Individual UCEs

The gene trees were constructed using ETE toolkit:

Preparatory step: installing Miniconda, following the instructions from http://etetoolkit.org/download/

The gene trees were estimated using RAxML:

ete3 build -w standard_raxml -n /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/FASTA/uce-3.fasta -o /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/FASTA/uce-3-tree

Automatically generate the corresponding command for each of the 230 alignment files in the directory:

cd /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/FASTA
ls

Copy the output to a text file:

# Get the 1st part of the command

locus_table <- read.table("Set1-individual-loci.txt")
n <- length(unlist(locus_table))

inputfile <- vector()
for(i in locus_table) {
  inputfile <- append(inputfile, paste("ete3 build -w standard_raxml -n /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/FASTA/", i, sep = ""))
}

# Length check

length(inputfile) - n

# Remove file endings from the locus names

names_and_endings <- as.character(as.vector(as.matrix(locus_table)))
locus_names <- vector()
for(i in names_and_endings) {
  locus_names <- append(locus_names, sapply(strsplit(i, split='.', fixed=TRUE), function(x) (x[1])))
}

# Get the 2nd part of the command

outputfile <- vector()
for(i in locus_names) {
  outputfile <- append(outputfile, paste(" -o /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/", i, sep=""))
}

# Length check

length(outputfile) - n

# Get the entire command

commands <- paste(inputfile, outputfile, "-tree &&", sep="")

# Length check

length(commands) - n

# Uncomment to print to file:
# write(commands, "gene-tree-analysis.sh")

#!/bin/bash was then inserted into the first line of the file to convert it into a shell script. The script was run as follows:

chmod 755 /Users/David/Grive/Alfaro_Lab/SortaDate/Locus_analysis/gene-tree-analysis.sh
/Users/David/Grive/Alfaro_Lab/SortaDate/Locus_analysis/gene-tree-analysis.sh

Write a new script that will copy the trees from the nested subdirectories to the same directory where the alignment files are located:

# 1st part of the command

copyfiles <- vector()
for(i in locus_names) {
  copyfiles <- append(copyfiles, paste("cp /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/", i, sep = ""))
}

# 2nd part of the command

subdirectories <- vector()
for(i in locus_names) {
  subdirectories <- append(subdirectories, paste("-tree/clustalo_default-none-none-raxml_default/", i, sep = ""))
}

# Get the entire command

copying <- paste(copyfiles, subdirectories, ".fasta.final_tree.nw /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/FASTA/ &&", sep="")

# Uncomment to print to file:
# write(copying, "copy-trees.sh")

Write a third script to rename the tree files so that they have the .tre file ending, which SortaDate looks for while searching its target directory:

# 1st part of the command

oldname <- vector()
for(i in locus_names) {
  oldname <- append(oldname, paste("mv /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/FASTA/", i, sep = ""))
}

# 2nd part of the command

newname <- vector()
for(i in locus_names) {
  newname <- append(newname, paste(".fasta.final_tree.nw /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/FASTA/", i, sep = ""))
}

# Get the entire command:

renametrees <- paste(oldname, newname, ".tre &&", sep = "")

# Uncomment to print to file:
write(renametrees, "rename-trees.sh")

Phase 1: get_var_length

python src/get_var_length.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/FASTA/ --flend .tre --outf Locus_analysis/var-uces --outg alepisaurus_ferox,ceratoscopelus_warmingii

Phase 2: get_bp_genetrees

python src/get_bp_genetrees.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/FASTA/ /Users/David/Grive/Alfaro_Lab/Acanthomorpha/12_no_outgroups_scheme_3.tre --flend .tre --outf Locus_analysis/bp-uces

Phase 3: combine_results

python src/combine_results.py Locus_analysis/var-uces Locus_analysis/bp-uces --outf Locus_analysis/comb-uces

Phase 4: get_good_genes

In order of descending priority: bipartition support, root-to-tip variance, tree length

python src/get_good_genes.py Locus_analysis/comb-uces --max 3 --order 3,1,2 --outf Locus_analysis/gg-uces-312
name root-to-tip_var treelength bipartition
uce-1184.tre 0.00510364 5.59114 0.491379310345
uce-383.tre 0.0094896 4.68126 0.465517241379
uce-1317.tre 0.0200846 8.06488 0.465517241379

In order of descending priority: bipartition support, tree length, root-to-tip variance

python src/get_good_genes.py Locus_analysis/comb-uces --max 3 --order 3,2,1 --outf Locus_analysis/gg-uces-321
name root-to-tip_var treelength bipartition
uce-1184.tre 0.00510364 5.59114 0.491379310345
uce-383.tre 0.0094896 4.68126 0.465517241379
uce-1317.tre 0.0200846 8.06488 0.465517241379

In order of descending priority: root-to-tip variance, tree length, bipartition support

python src/get_good_genes.py Locus_analysis/comb-uces --max 3 --order 1,2,3 --outf Locus_analysis/gg-uces-123
name root-to-tip_var treelength bipartition
uce-1075.tre 7.59021e-05 0.363795 0.0689655172414
uce-446.tre 0.000230387 0.897702 0.26724137931
uce-855.tre 0.000309407 1.03296 0.301724137931

In order of descending priority: root-to-tip variance, bipartition support, tree length

python src/get_good_genes.py Locus_analysis/comb-uces --max 3 --order 1,3,2 --outf Locus_analysis/gg-uces-132
name root-to-tip_var treelength bipartition
uce-1075.tre 7.59021e-05 0.363795 0.0689655172414
uce-446.tre 0.000230387 0.897702 0.26724137931
uce-855.tre 0.000309407 1.03296 0.301724137931

In order of descending priority: tree length, root-to-tip variance, bipartition support

python src/get_good_genes.py Locus_analysis/comb-uces --max 3 --order 2,1,3 --outf Locus_analysis/gg-uces-213
name root-to-tip_var treelength bipartition
uce-1075.tre 7.59021e-05 0.363795 0.0689655172414
uce-737.tre 0.000743068 0.814482 0.137931034483
uce-446.tre 0.000230387 0.897702 0.26724137931

In order of descending priority: tree length, bipartition support, root-to-tip variance

python src/get_good_genes.py Locus_analysis/comb-uces --max 3 --order 2,3,1 --outf Locus_analysis/gg-uces-231
name root-to-tip_var treelength bipartition
uce-1075.tre 7.59021e-05 0.363795 0.0689655172414
uce-737.tre 0.000743068 0.814482 0.137931034483
uce-446.tre 0.000230387 0.897702 0.26724137931

k-means partitions

Note that the largest, slowest-evolving partition (“ccf55a6ee6d62f840a124bcc0c98ecf5”; 132 kb) was excluded from the first round of analyses for computational reasons. Attempts to analyze it in RAxML after the remaining 31 analyses finished up were unsuccessful.

# Get the 1st part of the command

kmeans_table <- read.table("Set2-kmeans-partitions.txt")
n2 <- length(unlist(kmeans_table))

first <- vector()
for(i in kmeans_table) {
  first <- append(first, paste("ete3 build -w standard_raxml -n /Users/David/Grive/Alfaro_Lab/Acanthomorpha/k-means_partitions/", i, sep = ""))
}

# Length check

length(first) - n2

# Remove file endings from the locus names

no_endings <- as.character(as.vector(as.matrix(kmeans_table)))
kmeans_names <- vector()
for(i in no_endings) {
  kmeans_names <- append(kmeans_names, sapply(strsplit(i, split='.', fixed=TRUE), function(x) (x[1])))
}
kmeans_names

# Get the 2nd part of the command

second <- vector()
for(i in kmeans_names) {
  second <- append(second, paste(" -o /Users/David/Grive/Alfaro_Lab/Acanthomorpha/k-means_partitions/", i, sep=""))
}
second

# Length check

length(second) - n2

# Get the entire command

kmeansscript <- paste(first, second, "-tree &&", sep="")

# Length check

length(kmeansscript) - n2

# Uncomment to print to file:
# write(kmeansscript, "kmeans-analysis.sh")

Copy the trees into the directory containing the alignments:

# 1st part of the command

copytrees <- vector()
for(i in kmeans_names) {
  copytrees <- append(copytrees, paste("cp /Users/David/Grive/Alfaro_Lab/Acanthomorpha/k-means_partitions/", i, sep = ""))
}

# 2nd part of the command

locations <- vector()
for(i in kmeans_names) {
  locations <- append(locations, paste("-tree/clustalo_default-none-none-raxml_default/", i, sep = ""))
}

# Get the entire command

finalstep <- paste(copytrees, locations, ".phy.fasta.final_tree.nw /Users/David/Grive/Alfaro_Lab/Acanthomorpha/k-means_partitions/FASTA/ &&", sep="")

# Uncomment to print to file:
# write(finalstep, "copy-kmeans-trees.sh")

Change the tree names so that they correspond to the partition names:

# 1st part of the command

oldtreename <- vector()
for(i in kmeans_names) {
  oldtreename <- append(oldtreename, paste("mv /Users/David/Grive/Alfaro_Lab/Acanthomorpha/k-means_partitions/FASTA/", i, sep = ""))
}

# 2nd part of the command

newtreename <- vector()
for(i in kmeans_names) {
  newtreename <- append(newtreename, paste(".phy.fasta.final_tree.nw /Users/David/Grive/Alfaro_Lab/Acanthomorpha/k-means_partitions/FASTA/", i, sep = ""))
}

# Get the entire command:

changetreenames <- paste(oldtreename, newtreename, ".tre &&", sep = "")

# Uncomment to print to file:
# write(changetreenames, "rename-kmeans-trees.sh")

Finally, rename the partitions:

# 1st part of the command

oldpartition <- vector()
for(i in kmeans_names) {
  oldpartition <- append(oldpartition, paste("mv /Users/David/Grive/Alfaro_Lab/Acanthomorpha/k-means_partitions/FASTA/", i, sep = ""))
}

# 2nd part of the command

newpartition <- vector()
for(i in kmeans_names) {
  newpartition <- append(newpartition, paste(".phy.fasta /Users/David/Grive/Alfaro_Lab/Acanthomorpha/k-means_partitions/FASTA/", i, sep = ""))
}

# Get the entire command
  
renamepartitions <- paste(oldpartition, newpartition, ".fasta &&", sep = "")

# Uncomment to print to file:
# write(renamepartitions, "rename-partitions.sh")

Phase 1: get_var_length

python src/get_var_length.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/k-means_partitions/FASTA/ --flend .tre --outf var-kmeans --outg alepisaurus_ferox,ceratoscopelus_warmingii

Phase 2: get_bp_genetrees

python src/get_bp_genetrees.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/k-means_partitions/FASTA/ /Users/David/Grive/Alfaro_Lab/Acanthomorpha/12_no_outgroups_scheme_3.tre --flend .tre --outf bp-kmeans

Phase 3: combine_results

python src/combine_results.py var-kmeans bp-kmeans --outf comb-kmeans

Phase 4: get_good_genes

In order of descending priority: bipartition support, root-to-tip variance, tree length

name root-to-tip_var treelength bipartition
8be7c94dcf2d71970c663f6710af40d7.tre 0.0878182 19.8649 0.560344827586
f495e4e0f2f9bbbf091f778067b062f4.tre 0.0230499 11.0301 0.508620689655
0061a6137d1029978876fe13239f57bc.tre 0.0205424 11.2184 0.5

In order of descending priority: bipartition support, tree length, root-to-tip variance

name root-to-tip_var treelength bipartition
8be7c94dcf2d71970c663f6710af40d7.tre 0.0878182 19.8649 0.560344827586
f495e4e0f2f9bbbf091f778067b062f4.tre 0.0230499 11.0301 0.508620689655
0061a6137d1029978876fe13239f57bc.tre 0.0205424 11.2184 0.5

In order of descending priority: root-to-tip variance, tree length, bipartition support

name root-to-tip_var treelength bipartition
4ff6dce0b7cebc9428b4b77e79356484.tre 0.000241636 1.15434 0.0258620689655
589d12bf910a9937d941aa4c836ad87d.tre 0.000935835 2.0223 0.336206896552
fe7aa5d9de0f5f51ef6365ef3b7f1e06.tre 0.00125304 2.60574 0.0603448275862

In order of descending priority: root-to-tip variance, bipartition support, tree length

name root-to-tip_var treelength bipartition
4ff6dce0b7cebc9428b4b77e79356484.tre 0.000241636 1.15434 0.0258620689655
589d12bf910a9937d941aa4c836ad87d.tre 0.000935835 2.0223 0.336206896552
fe7aa5d9de0f5f51ef6365ef3b7f1e06.tre 0.00125304 2.60574 0.0603448275862

In order of descending priority: tree length, root-to-tip variance, bipartition support

name root-to-tip_var treelength bipartition
4ff6dce0b7cebc9428b4b77e79356484.tre 0.000241636 1.15434 0.0258620689655
589d12bf910a9937d941aa4c836ad87d.tre 0.000935835 2.0223 0.336206896552
fe7aa5d9de0f5f51ef6365ef3b7f1e06.tre 0.00125304 2.60574 0.0603448275862

In order of descending priority: tree length, bipartition support, root-to-tip variance

name root-to-tip_var treelength bipartition
4ff6dce0b7cebc9428b4b77e79356484.tre 0.000241636 1.15434 0.0258620689655
589d12bf910a9937d941aa4c836ad87d.tre 0.000935835 2.0223 0.336206896552
fe7aa5d9de0f5f51ef6365ef3b7f1e06.tre 0.00125304 2.60574 0.0603448275862

50-bp Chunks

A bash script was written to automate the following actions:

  1. Create a new directory nested in /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/FASTA for each locus
  2. Split the alignments for each locus into individual sequences, and these into 50-bp chunks. This step was performed using a custom Python script obtained from http://www.reddit.com/r/bioinformatics/comments/1u8yc7/looking_for_a_script_that_will_split_dna/ceg8rav/?st=j0tbjfco&sh=1e300055.

      '''
      Splits all sequences within a multi-fasta file into chunks of a specified size.
    
      Fasta header information is retained with each split sequence - its position in
      the original is appended to the id. Single-line and multi-line fasta files are
      supported. Prints to stout, so pipe to a file to store the result.
    
      Usage:
      python splitter.py <filename> <chunksize>
      python splitter.py myfile.fa 100
      '''
    
      from __future__ import print_function
      from sys import argv, version
    
      if version[0] == '2':
          from itertools import izip_longest as zl
      else:
          from itertools import zip_longest as zl
    
      chunksize = int(argv[2])
    
      def writeseq(header, seq):
          for i, chunk in enumerate(zl(*[iter(seq)]*chunksize, fillvalue='')):
              print(header + '_{}bp'.format(i*chunksize))
              print(''.join(chunk))
    
      with open(argv[1]) as f:
          header, seq = f.readline().rstrip(), ''
          for l in f:
              if l[0] != '>':
                  seq += l.rstrip()
              else:
                  writeseq(header, seq)
                  header, seq = l.rstrip(), ''
          writeseq(header, seq)
  3. For each locus, join the individual sequences chunk-wise (i.e., make a single fasta file for all taxa and sites 0 to 50, another one for all taxa and sites 51 to 100, etc.):

    library(dplyr)
    
    # Alternating rows (name, sequence, name, sequence) go to two different columns, so that
    # each sequence is correctly assigned to its respective taxon:
    
    split_seqs <- read.table("split.txt")
    odd <- as.vector(split_seqs[seq(1, nrow(split_seqs), 2), ])
    even <- as.vector(split_seqs[seq(2, nrow(split_seqs), 2), ])
    odd_name <- "taxon"
    even_name <- "sequence"
    split_seqs_new <- data.frame(odd, even)
    names(split_seqs_new) <- c(odd_name, even_name)
    
    # Determine how long the locus is (i.e., how many 50-bp chunks it has been split into).
    # This can be done by counting the number of occurrences of a single taxon name. 
    # In principle, any name could be used, but since not all of the UCEs include
    # all of the taxa, it is advisable to choose a taxon common to all the loci.
    
    n <- length(unique(grep("chaetodon_kleinii", split_seqs_new[,1], value = TRUE)))
    
    # Create a vector of strings that can filter taxon names according to the base pair range
    # tag attached to their end
    
    chunks <- vector()
    for(i in 0:(n-1)) {
      chunks <- append(chunks, paste("_", i*50, "bp", sep = ""))
    }
    
    # Create a list of data frames. Each element of the list represents a base pair range
    # and consists of a data frame containing both the "taxon" and "sequence" columns of
    # split_seqs_new
    
    partition <- list()
    for(i in 1:length(chunks)) {
      partition[[i]] <- data.frame(filter(split_seqs_new, 
                                          grepl(as.character(chunks[i]), taxon)))
    }
    
    # Create a matrix whose columns represents individual chunks (i.e., base pair ranges)
    # and whose rows have the structure of the original split_seqs data frame -- i.e., name,
    # sequence, name, sequence:
    
    chunkmatrix <- matrix(ncol = length(partition), 
                          nrow = 2*(nrow(split_seqs_new)/length(partition)))
    for(i in 1:length(partition)) {
      for(j in 1:nrow(partition[[i]])) {
        chunkmatrix[(2*j - 1), i] <- as.character(partition[[i]][j, "taxon"])
        chunkmatrix[2*j, i] <- as.character(partition[[i]][j, "sequence"])
      }
    }
    
    # Print the resulting fasta files!
    
    for(i in 1:ncol(chunkmatrix)) {
      write(chunkmatrix[,i], paste("chunk_", i, ".fasta", sep = ""))
    }
  4. Add a locus-indicating prefix to all the chunks of a given UCE:

    find *.fasta -maxdepth 0 ! -path . -exec mv {} uce-1005_{} \;
  5. Copy the resulting fasta files into a single directory.

The contents of the directory were then summarized as follows:

cd /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks/
ls > /Users/David/Grive/Alfaro_Lab/SortaDate/Set3-50bp-chunks.txt

Now, change the taxon names in the chunk FASTA files so that they correspond to the names in the reference tree. First, create a file with the names of all the chunk files:

cd /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks/
ls *.fasta > /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks/chunklist.txt
a <- read.table("/Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks/chunklist.txt")

x <- vector()
for(i in 1:nrow(a)) {
  x <- append(x, paste("/Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks/", a[i,], sep = ""))
}

for(i in 1:length(x)) {
  c <- read.table(print(x[i]), stringsAsFactors = FALSE)
  d <- vector()
  for(j in 1:(nrow(c)/2)) {
    d[j] <- as.character(c[(2*j-1),])
  }
  
  e <- vector()
  for(j in 1:length(d)) {
    e[j] <- paste(sapply(strsplit(d[j], split="_", fixed=TRUE), function(x) (x[1])), 
                  "_", 
                  sapply(strsplit(d[j], split="_", fixed=TRUE), function(x) (x[2])),
                  sep = "")
  }
  
  for(j in 1:(nrow(c)/2)) {
    c[(2*j-1),] <- e[j]
  }
  write(as.matrix(c), print(x[i]), ncolumns=1)
}

A script was generated to analyze all of the alignment in the directory using RAxML:

chunk_table <- read.table("Set3-50bp-chunks.txt")

newfolders <- vector()
for(i in chunk_table) {
  newfolders <- append(newfolders, paste("ete3 build -w standard_raxml -n /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks/", i, sep=""))
}

with_endings <- as.character(as.vector(as.matrix(chunk_table)))
chunk_names <- vector()
for(i in with_endings) {
  chunk_names <- append(chunk_names, sapply(strsplit(i, split='.', fixed=TRUE), function(x) (x[1])))
}

tree_location <- vector()
for(i in chunk_names) {
  tree_location <- append(tree_location, paste(" -o /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/", i, sep=""))
}

analyzechunks <- paste(newfolders, tree_location, sep="")
write(analyzechunks, "chunk-analysis.sh")

The resulting tree files were then copied into the directory containing the alignments:

copyfrom1 <- vector()
for(i in chunk_names) {
  copyfrom1 <- append(copyfrom1, paste("cp /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/", i, sep = ""))
}

copyfrom2 <- vector()
for(i in chunk_names) {
  copyfrom2 <- append(copyfrom2, paste("/clustalo_default-none-none-raxml_default/", i, sep = ""))
}

copyto <- paste(copyfrom1, copyfrom2, ".fasta.final_tree.nw /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks/", sep="")

write(copyto, "copy-chunk-trees.sh")

Rename the trees:

brew install rename
rename -S .fasta.final_tree.nw .tre *.fasta.final_tree.nw

Running ls *.fasta | wc -l and ls *.nw | wc -l in the directory showed that out of 1826 chunk FASTA files, no more than 1070 had tree files associated with them.

Phase 1: get_var_length

python src/get_var_length.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks/ --flend .tre --outf Chunk_analysis/var-chunks --outg alepisaurus_ferox,ceratoscopelus_warmingii

Phase 2: get_bp_genetrees

python src/get_bp_genetrees.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks/ /Users/David/Grive/Alfaro_Lab/Acanthomorpha/12_no_outgroups_scheme_3.tre --flend .tre --outf Chunk_analysis/bp-chunks

Phase 3: combine_results

python src/combine_results.py Chunk_analysis/var-chunks Chunk_analysis/bp-chunks --outf Chunk_analysis/comb-chunks

Calculate the Robinson-Foulds distances of the individual chunk trees from the reference tree using the Python script below:

import os, uuid
from ete3 import Tree

t2 = Tree("/Users/David/Grive/Alfaro_Lab/Acanthomorpha/12_no_outgroups_scheme_3.tre")
for file in os.listdir("/Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks/"):
    if file.endswith(".tre"):
        t1 = Tree(file)
        try:
            rf = t1.robinson_foulds(t2)
            print str(file), (rf[0])
        except:
            pass

Step 1 of filtering: remove trees of zero length

No effect: the minimum observed tree length is non-zero and equal to:

## [1] 8.80081e-05

Phase 4: get_good_genes

In order of descending priority: bipartition support, root-to-tip variance, tree length

name root-to-tip_var treelength bipartition
uce-157_chunk_7.tre 0.176994 14.918 0.258620689655
uce-126_chunk_3.tre 0.104056 14.0824 0.224137931034
uce-323_chunk_2.tre 1.16412 48.0104 0.224137931034
## [1] "The Robinson-Foulds distances of the three bets chunks from the reference tree are as follows:"
##                   chunk  rf
## 321 uce-157_chunk_7.tre 164
##                   chunk  rf
## 219 uce-126_chunk_3.tre 166
##                   chunk  rf
## 443 uce-323_chunk_2.tre 168

The chunk with the minimum RF distance from the reference tree:

##                   chunk  rf
## 321 uce-157_chunk_7.tre 164

In order of descending priority: bipartition support, tree length, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-157_chunk_7.tre 0.176994 14.918 0.258620689655
uce-126_chunk_3.tre 0.104056 14.0824 0.224137931034
uce-323_chunk_2.tre 1.16412 48.0104 0.224137931034

In order of descending priority: root-to-tip variance, tree length, bipartition support

name root-to-tip_var treelength bipartition
uce-1243_chunk_4.tre 1.05801e-11 0.0215722 0.0
uce-600_chunk_3.tre 1.28251e-11 0.0205929 0.00862068965517
uce-737_chunk_4.tre 2.16826e-11 0.0215345 0.0

In order of descending priority: root-to-tip variance, bipartition support, tree length

name root-to-tip_var treelength bipartition
uce-1243_chunk_4.tre 1.05801e-11 0.0215722 0.0
uce-600_chunk_3.tre 1.28251e-11 0.0205929 0.00862068965517
uce-737_chunk_4.tre 2.16826e-11 0.0215345 0.0

In order of descending priority: tree length, root-to-tip variance, bipartition support

name root-to-tip_var treelength bipartition
uce-855_chunk_6.tre 3.60737e-11 8.80081e-05 0.0
uce-413_chunk_4.tre 3.51022e-06 0.0205344 0.0
uce-600_chunk_3.tre 1.28251e-11 0.0205929 0.00862068965517

In order of descending priority: tree length, bipartition support, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-855_chunk_6.tre 3.60737e-11 8.80081e-05 0.0
uce-413_chunk_4.tre 3.51022e-06 0.0205344 0.0
uce-600_chunk_3.tre 1.28251e-11 0.0205929 0.00862068965517

50-bp chunks: SH-like of 75% or above

  1. Copy all the tree files to a new directory:

    cd /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa
    mkdir Chunks_75
    cp Chunks/*.tre Chunks_75
  2. Collapse all the nodes with SH-like support values of less than 75% using the following Python script:

    import os, uuid
    from ete3 import Tree
    
    for file in os.listdir("/Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks"):
        if file.endswith(".tre"):
            outname = "/Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks_75/" + str(file)
            t = Tree(file, format=0)
    
            print t.get_ascii(attributes=['support', 'name'])
    
            for node in t.get_descendants():
                if not node.is_leaf() and node.support <= 0.75:
                    node.delete()
    
            print t.get_ascii(attributes=['support', 'name'])
    
            t.write(format=0, outfile=outname)
  3. Copy the fasta alignments to the same directory:

    cp Chunks/*.fasta Chunks_75

Phase 1: get_var_length

python src/get_var_length.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks_75/ --flend .tre --outf Chunk75_analysis/var-chunks75 --outg alepisaurus_ferox,ceratoscopelus_warmingii

Phase 2: get_bp_genetrees

python src/get_bp_genetrees.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks_75/ /Users/David/Grive/Alfaro_Lab/Acanthomorpha/12_no_outgroups_scheme_3.tre --flend .tre --outf Chunk75_analysis/bp-chunks75

Phase 3: combine_results

python src/combine_results.py Chunk75_analysis/var-chunks75 Chunk75_analysis/bp-chunks75 --outf Chunk75_analysis/comb-chunks75

A script was written to delete all lines containing NAs, as well as all lines that only contained an entry for bipartition support but none for root-to-tip branch length variance or tree length. This was accomplished by filling these partially empty lines with NAs in the first step and deleting them in the second step:

combchunks75 <- read.table("/Users/David/Grive/Alfaro_Lab/SortaDate/Chunk75_analysis/comb-chunks75", fill = TRUE)
filtered <- na.omit(combchunks75)
write.table(filtered,
      "/Users/David/Grive/Alfaro_Lab/SortaDate/Chunk75_analysis/comb-chunks75-filtered",
      sep = "\t",
      quote = FALSE,
      row.names = FALSE,
      col.names = FALSE)

The length of the filtered comb file is 901 lines, meaning that SortaDate failed to obtain root-to-tip variance and/or tree length for 169 trees.

Step 1 of filtering: remove trees of zero length

No effect: the minimum observed tree length is non-zero and equal to:

## [1] 0.0202802

Note that this result is orders of magnitude larger than those observed in the other five chunk datasets.

Phase 4: get_good_genes

In order of descending priority: bipartition support, root-to-tip variance, tree length

name root-to-tip_var treelength bipartition
uce-157_chunk_7.tre 0.163848 13.508 0.198275862069
uce-126_chunk_3.tre 0.101031 13.9331 0.181034482759
uce-323_chunk_2.tre 0.459664 40.9379 0.146551724138
## [1] "The Robinson-Foulds distances of the three best chunks from the reference tree are as follows:"
##                   chunk  rf
## 284 uce-157_chunk_7.tre 126
##                   chunk  rf
## 194 uce-126_chunk_3.tre 114
##                   chunk  rf
## 391 uce-323_chunk_2.tre 125

The chunk with the minimum RF distance from the reference tree:

##                    chunk  rf
## 189 uce-1263_chunk_2.tre 106

In order of descending priority: bipartition support, tree length, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-157_chunk_7.tre 0.163848 13.508 0.198275862069
uce-126_chunk_3.tre 0.101031 13.9331 0.181034482759
uce-323_chunk_2.tre 0.459664 40.9379 0.146551724138

In order of descending priority: root-to-tip variance, tree length, bipartition support

name root-to-tip_var treelength bipartition
uce-815_chunk_8.tre 0.0 0.0202802 0.0
uce-359_chunk_7.tre 0.0 0.0203467 0.00862068965517
uce-160_chunk_3.tre 0.0 0.0204413 0.0

In order of descending priority: root-to-tip variance, bipartition support, tree length

name root-to-tip_var treelength bipartition
uce-200_chunk_2.tre 0.0 0.0317025 0.0344827586207
uce-129_chunk_5.tre 0.0 0.0211935 0.0258620689655
uce-1062_chunk_5.tre 0.0 0.113443 0.0258620689655

In order of descending priority: tree length, root-to-tip variance, bipartition support

name root-to-tip_var treelength bipartition
uce-815_chunk_8.tre 0.0 0.0202802 0.0
uce-359_chunk_7.tre 0.0 0.0203467 0.00862068965517
uce-160_chunk_3.tre 0.0 0.0204413 0.0

In order of descending priority: tree length, bipartition support, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-815_chunk_8.tre 0.0 0.0202802 0.0
uce-359_chunk_7.tre 0.0 0.0203467 0.00862068965517
uce-160_chunk_3.tre 0.0 0.0204413 0.0

50-bp chunks: SH-like of 90% or above

(The first three steps were identical to those described above.)

Phase 1: get_var_length

python src/get_var_length.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks_90/ --flend .tre --outf Chunk90_analysis/var-chunks90 --outg alepisaurus_ferox,ceratoscopelus_warmingii

Phase 2: get_bp_genetrees

python src/get_bp_genetrees.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-114-taxa/Chunks_90/ /Users/David/Grive/Alfaro_Lab/Acanthomorpha/12_no_outgroups_scheme_3.tre --flend .tre --outf Chunk90_analysis/bp-chunks90

Phase 3: combine_results

python src/combine_results.py Chunk90_analysis/var-chunks90 Chunk90_analysis/bp-chunks90 --outf Chunk90_analysis/comb-chunks90

Delete the lines with NAs:

combchunks90 <- read.table("/Users/David/Grive/Alfaro_Lab/SortaDate/Chunk90_analysis/comb-chunks90", fill = TRUE)
filtered <- na.omit(combchunks90)
write.table(filtered,
      "/Users/David/Grive/Alfaro_Lab/SortaDate/comb-chunks90-filtered",
      sep = "\t",
      quote = FALSE,
      row.names = FALSE,
      col.names = FALSE)

The length of the filtered comb file is 536 lines, meaning that SortaDate failed to obtain root-to-tip variance and/or tree length for 534 (almost 50%) trees.

Step 1 of filtering: remove trees of zero length

No effect: the minimum observed tree length is non-zero and equal to:

## [1] 8.71304e-07

Phase 4: get_good_genes

In order of descending priority: bipartition support, root-to-tip variance, tree length

name root-to-tip_var treelength bipartition
uce-1317_chunk_8.tre 0.0166339 15.2225 0.0689655172414
uce-120_chunk_6.tre 0.187599 4.6936 0.0689655172414
uce-323_chunk_2.tre 0.325194 35.4206 0.0689655172414
## [1] "The Robinson-Foulds distances of the three best chunks from the reference tree are as follows:"
##                    chunk  rf
## 159 uce-1317_chunk_8.tre 111
##                  chunk  rf
## 81 uce-120_chunk_6.tre 114
##                   chunk  rf
## 248 uce-323_chunk_2.tre 107

The chunk with the minimum RF distance from the reference tree:

##                   chunk  rf
## 235 uce-267_chunk_7.tre 106

In order of descending priority: bipartition support, tree length, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-120_chunk_6.tre 0.187599 4.6936 0.0689655172414
uce-1317_chunk_8.tre 0.0166339 15.2225 0.0689655172414
uce-323_chunk_2.tre 0.325194 35.4206 0.0689655172414

In order of descending priority: root-to-tip variance, tree length, bipartition support

name root-to-tip_var treelength bipartition
uce-537_chunk_3.tre 0.0 8.71304e-07 0.00862068965517
uce-84_chunk_5.tre 0.0 9.84452e-07 0.0172413793103
uce-739_chunk_2.tre 0.0 2.2254e-06 0.0344827586207

In order of descending priority: root-to-tip variance, bipartition support, tree length

name root-to-tip_var treelength bipartition
uce-739_chunk_2.tre 0.0 2.2254e-06 0.0344827586207
uce-832_chunk_4.tre 0.0 1.05876 0.0258620689655
uce-84_chunk_5.tre 0.0 9.84452e-07 0.0172413793103

In order of descending priority: tree length, root-to-tip variance, bipartition support

name root-to-tip_var treelength bipartition
uce-537_chunk_3.tre 0.0 8.71304e-07 0.00862068965517
uce-84_chunk_5.tre 0.0 9.84452e-07 0.0172413793103
uce-739_chunk_2.tre 0.0 2.2254e-06 0.0344827586207

In order of descending priority: tree length, bipartition support, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-537_chunk_3.tre 0.0 8.71304e-07 0.00862068965517
uce-84_chunk_5.tre 0.0 9.84452e-07 0.0172413793103
uce-739_chunk_2.tre 0.0 2.2254e-06 0.0344827586207

50-bp chunks from the 75%-complete dataset

A bash script used to split the UCE loci into 50-bp chunks was identical to that used above, with the exception of step 3 (joining all taxa represented by a given chunk into a new FASTA file), for which the following code was used:

library(dplyr)

# Alternating rows (name, sequence, name, sequence) go to two different columns, so that
# each sequence is correctly assigned to its respective taxon:

split_seqs <- read.table("split.txt")
odd <- as.vector(split_seqs[seq(1, nrow(split_seqs), 2), ])
even <- as.vector(split_seqs[seq(2, nrow(split_seqs), 2), ])
odd_name <- "taxon"
even_name <- "sequence"
split_seqs_new <- data.frame(odd, even)
names(split_seqs_new) <- c(odd_name, even_name)

# Determine how long the locus is (i.e., how many 50-bp chunks it has been split into).
# This can be done by counting the number of occurrences of a single taxon name. 
# In principle, any name could be used, but since no taxon appears to be common to all
# the loci, the code below grabs the first taxon name appearing in the fasta file and
# counts its occurrences.

m <- paste(sapply(strsplit(as.character(split_seqs_new[1,1]), split="_", fixed=TRUE),
                  function(x) (x[1])), 
           "_", sapply(strsplit(as.character(split_seqs_new[1,1]), split="_", fixed=TRUE),                         function(x) (x[2])),
           sep = "")
n <- length(unique(grep(m, split_seqs_new[,1], value = TRUE)))

# Create a vector of strings that can filter taxon names according to the base pair range
# tag attached to their end

chunks <- vector()
for(i in 0:(n-1)) {
  chunks <- append(chunks, paste("_", i*50, "bp", sep = ""))
}

# Create a list of data frames. Each element of the list represents a base pair range
# and consists of a data frame containing both the "taxon" and "sequence" columns of
# split_seqs_new

partition <- list()
for(i in 1:length(chunks)) {
partition[[i]] <- data.frame(filter(split_seqs_new, 
                                    grepl(as.character(chunks[i]), taxon)))
}

# Create a matrix whose columns represents individual chunks (i.e., base pair ranges)
# and whose rows have the structure of the original split_seqs data frame -- i.e., name,
# sequence, name, sequence:

chunkmatrix <- matrix(ncol = length(partition), 
                      nrow = 2*(nrow(split_seqs_new)/length(partition)))
for(i in 1:length(partition)) {
  for(j in 1:nrow(partition[[i]])) {
    chunkmatrix[(2*j - 1), i] <- as.character(partition[[i]][j, "taxon"])
    chunkmatrix[2*j, i] <- as.character(partition[[i]][j, "sequence"])
  }
}

# Print the resulting fasta files!

for(i in 1:ncol(chunkmatrix)) {
  write(chunkmatrix[,i], paste("chunk_", i, ".fasta", sep = ""))
}

The taxon names in the chunk FASTA files were then stripped of the chunk-indicating suffixes so as to correspond with the names used in the reference tree, and a script was used to analyze all the chunks using RAxML. The resulting trees were then copied into the directory containing the chunks. Out of 6543 chunk FASTA files, only 4237 had tree files associated with them, suggesting that RAxML failed to infer a tree for a given chunk in approx. 35% of cases.

Phase 1: get_var_length

python src/get_var_length.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-90-taxa/Chunks/ --flend .tre --outf Min-90-chunk_analysis/var-min-90-chunks --outg alepisaurus_ferox,ceratoscopelus_warmingii

Running the script produced 1029 warnings about either Alepisaurus ferox or Ceratoscopelus warmingii missing from the chunk tree. Despite this, the tree length and root-to-tip variance was computed for all the 4237 trees.

Phase 2: get_bp_genetrees

python src/get_bp_genetrees.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-90-taxa/Chunks/ /Users/David/Grive/Alfaro_Lab/Acanthomorpha/12_no_outgroups_scheme_3.tre --flend .tre --outf Min-90-chunk_analysis/bp-min-90-chunks

Phase 3: combine_results

python src/combine_results.py Min-90-chunk_analysis/var-min-90-chunks Min-90-chunk_analysis/bp-min-90-chunks --outf Min-90-chunk_analysis/comb-min-90-chunks

Phase 4: get_good_genes

In order of descending priority: bipartition support, root-to-tip variance, tree length

name root-to-tip_var treelength bipartition
uce-157_chunk_7.tre 0.176994 14.918 0.258620689655
uce-1244_chunk_1.tre 0.0534134 11.1586 0.224137931034
uce-126_chunk_3.tre 0.104056 14.0824 0.224137931034
## [1] "The Robinson-Foulds distances of the three best chunks from the reference tree are as follows:"
##                    chunk  rf
## 1229 uce-157_chunk_7.tre 164
##                    chunk  rf
## 822 uce-1244_chunk_1.tre 158
##                   chunk  rf
## 907 uce-126_chunk_3.tre 166

The chunk with the minimum RF distance from the reference tree:

##                  chunk  rf
## 6 uce-1000_chunk_6.tre 136

In order of descending priority: bipartition support, tree length, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-157_chunk_7.tre 0.176994 14.918 0.258620689655
uce-1244_chunk_1.tre 0.0534134 11.1586 0.224137931034
uce-126_chunk_3.tre 0.104056 14.0824 0.224137931034

In order of descending priority: root-to-tip variance, tree length, bipartition support

name root-to-tip_var treelength bipartition
uce-956_chunk_5.tre 2.3002e-12 8.56621e-05 0.0
uce-393_chunk_3.tre 5.54758e-12 9.22463e-05 0.0
uce-921_chunk_4.tre 5.8998e-12 7.89216e-05 0.0

In order of descending priority: root-to-tip variance, bipartition support, tree length

name root-to-tip_var treelength bipartition
uce-956_chunk_5.tre 2.3002e-12 8.56621e-05 0.0
uce-393_chunk_3.tre 5.54758e-12 9.22463e-05 0.0
uce-921_chunk_4.tre 5.8998e-12 7.89216e-05 0.0

In order of descending priority: tree length, root-to-tip variance, bipartition support

name root-to-tip_var treelength bipartition
uce-921_chunk_4.tre 5.8998e-12 7.89216e-05 0.0
uce-1173_chunk_3.tre 1.96604e-11 8.52591e-05 0.0
uce-956_chunk_5.tre 2.3002e-12 8.56621e-05 0.0

In order of descending priority: tree length, bipartition support, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-921_chunk_4.tre 5.8998e-12 7.89216e-05 0.0
uce-1173_chunk_3.tre 1.96604e-11 8.52591e-05 0.0
uce-956_chunk_5.tre 2.3002e-12 8.56621e-05 0.0

Step 1 of filtering: remove trees of zero length

No effect: the minimum observed tree length is non-zero and equal to:

## [1] 7.89216e-05
## Number of trees after filtering: 4237
## Percentage of trees after filtering: 100%

Step 2 of filtering: remove the shortest and longest 10% of trees:

## Original min. tree length: 7.89216e-05
## Original max. tree length: 214.974
## Min. tree length after step 2: 0.285338
## Max. tree length after step 2: 24.1424
## Number of trees after step 2: 3389
## Percentage of trees after step 2: 79.98584%

Step 3 of filtering: remove the most unclocklike third of loci

## Max. root-to-tip variance from step 2: 2.72349
## Max. root-to-tip variance after step 3: 0.00905244
## Number of trees after step 3: 2270
## Percentage of trees after step 3: 53.57564%

Step 4 of filtering: keep the loci in the upper 10% of bipartition support

## Min. bipartition support from step 3: 0
## Min. bipartition support after step 4: 0.1034483
## Number of trees after step 4: 170
## Percentage of trees after step 4: 4.012273%

50-bp chunks from the 75%-complete dataset: SH-like of 75% or above

The nodes with SH-like support values below 75% were collapsed using the script above. The corresponding FASTA files were copied into the resulting directory, and SortaDate was run as follows:

Phase 1: get_var_length

python src/get_var_length.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-90-taxa/Chunks_75/ --flend .tre --outf Min-90-chunk75_analysis/var-min-90-chunks75 --outg alepisaurus_ferox,ceratoscopelus_warmingii

Running the script resulted in occasional segmentation faults as well as “this really only works with nexus or newick” warning messages. The resulting file had the full number of lines (4237: one for each tree), but some of them were blank and others contained NAs.

Phase 2: get_bp_genetrees

python src/get_bp_genetrees.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-90-taxa/Chunks_75/ /Users/David/Grive/Alfaro_Lab/Acanthomorpha/12_no_outgroups_scheme_3.tre --flend .tre --outf Min-90-chunk75_analysis/bp-min-90-chunks75

Phase 3: combine_results

python src/combine_results.py Min-90-chunk75_analysis/var-min-90-chunks75 Min-90-chunk75_analysis/bp-min-90-chunks75 --outf Min-90-chunk75_analysis/comb-min-90-chunks75

Delete the lines with NAs:

comb90chunks75 <- read.table("/Users/David/Grive/Alfaro_Lab/SortaDate/Min-90-chunk75_analysis/comb-min-90-chunks75", fill = TRUE)
filtered <- na.omit(comb90chunks75)
write.table(filtered,
      "/Users/David/Grive/Alfaro_Lab/SortaDate/Min-90-chunk75_analysis/comb-min-90-chunks75-filtered",
      sep = "\t",
      quote = FALSE,
      row.names = FALSE,
      col.names = FALSE)
nrow(filtered)

The length of the filtered comb file is 3589 lines, meaning that SortaDate failed to obtain root-to-tip variance and/or tree length for 648 trees.

Phase 4: get_good_genes

In order of descending priority: bipartition support, root-to-tip variance, tree length

name root-to-tip_var treelength bipartition
uce-157_chunk_7.tre 0.163848 13.508 0.198275862069
uce-1244_chunk_1.tre 0.040797 10.3002 0.181034482759
uce-126_chunk_3.tre 0.101031 13.9331 0.181034482759
## [1] "The Robinson-Foulds distances of the three best chunks from the reference tree are as follows:"
##                    chunk  rf
## 1064 uce-157_chunk_7.tre 126
##                    chunk  rf
## 710 uce-1244_chunk_1.tre 111
##                   chunk  rf
## 788 uce-126_chunk_3.tre 114

The chunk with the minimum RF distance from the reference tree:

##                    chunk rf
## 2016 uce-463_chunk_2.tre 85

In order of descending priority: bipartition support, tree length, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-157_chunk_7.tre 0.163848 13.508 0.198275862069
uce-1244_chunk_1.tre 0.040797 10.3002 0.181034482759
uce-126_chunk_3.tre 0.101031 13.9331 0.181034482759

In order of descending priority: root-to-tip variance, tree length, bipartition support

name root-to-tip_var treelength bipartition
uce-1175_chunk_4.tre 0.0 8.48388e-07 0.00862068965517
uce-547_chunk_2.tre 0.0 1.18678e-06 0.0172413793103
uce-815_chunk_8.tre 0.0 0.0202802 0.0

In order of descending priority: root-to-tip variance, bipartition support, tree length

name root-to-tip_var treelength bipartition
uce-67_chunk_4.tre 0.0 0.0214551 0.0603448275862
uce-200_chunk_2.tre 0.0 0.0317025 0.0344827586207
uce-525_chunk_3.tre 0.0 0.020706 0.0258620689655

In order of descending priority: tree length, root-to-tip variance, bipartition support

name root-to-tip_var treelength bipartition
uce-1175_chunk_4.tre 0.0 8.48388e-07 0.00862068965517
uce-547_chunk_2.tre 0.0 1.18678e-06 0.0172413793103
uce-815_chunk_8.tre 0.0 0.0202802 0.0

In order of descending priority: tree length, bipartition support, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-1175_chunk_4.tre 0.0 8.48388e-07 0.00862068965517
uce-547_chunk_2.tre 0.0 1.18678e-06 0.0172413793103
uce-815_chunk_8.tre 0.0 0.0202802 0.0

Step 1 of filtering: remove trees of zero length

No effect: the minimum observed tree length is non-zero and equal to:

## [1] 8.48388e-07
## Number of trees after filtering: 3589
## Percentage of trees after filtering: 100%

Step 2 of filtering: remove the shortest and longest 10% of trees:

## Original min. tree length: 8.48388e-07
## Original max. tree length: 209.395
## Min. tree length after step 2: 0.124261
## Max. tree length after step 2: 19.679
## Number of trees after step 2: 2871
## Percentage of trees after step 2: 79.99443%

Step 3 of filtering: remove the most unclocklike third of loci

## Max. root-to-tip variance from step 2: 6.66986
## Max. root-to-tip variance after step 3: 0.00645959
## Number of trees after step 3: 1923
## Percentage of trees after step 3: 53.58038%

Step 4 of filtering: keep the loci in the upper 10% of bipartition support

## Min. bipartition support from step 3: 0
## Min. bipartition support after step 4: 0.06034483
## Number of trees after step 4: 169
## Percentage of trees after step 4: 4.708833%

50-bp chunks from the 75%-complete dataset: SH-like of 90% or above

Phase 1: get_var_length

python src/get_var_length.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-90-taxa/Chunks_90/ --flend .tre --outf Min-90-chunk90_analysis/var-min-90-chunks90 --outg alepisaurus_ferox,ceratoscopelus_warmingii

As in the previous case, the command led to a number of segfaults and “this really only works with nexus or newick” warnings, corresponding to lines in the resulting file that were either incomplete or included NAs.

Phase 2: get_bp_genetrees

python src/get_bp_genetrees.py /Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-90-taxa/Chunks_90/ /Users/David/Grive/Alfaro_Lab/Acanthomorpha/12_no_outgroups_scheme_3.tre --flend .tre --outf Min-90-chunk90_analysis/bp-min-90-chunks90

Phase 3: combine_results

python src/combine_results.py Min-90-chunk90_analysis/var-min-90-chunks90 Min-90-chunk90_analysis/bp-min-90-chunks90 --outf Min-90-chunk90_analysis/comb-min-90-chunks90

Delete the lines with NAs:

comb90chunks90 <- read.table("/Users/David/Grive/Alfaro_Lab/SortaDate/Min-90-chunk90_analysis/comb-min-90-chunks90", fill = TRUE)
filtered <- na.omit(comb90chunks90)
write.table(filtered,
      "/Users/David/Grive/Alfaro_Lab/SortaDate/Min-90-chunk90_analysis/comb-min-90-chunks90-filtered",
      sep = "\t",
      quote = FALSE,
      row.names = FALSE,
      col.names = FALSE)
nrow(filtered)

The length of the filtered comb file is 2319 lines, meaning that SortaDate failed to obtain root-to-tip variance and/or tree length for 1918 trees.

Phase 4: get_good_genes

In order of descending priority: bipartition support, root-to-tip variance, tree length

name root-to-tip_var treelength bipartition
uce-625_chunk_10.tre 0.0675117 0.706842 0.0775862068966
uce-1317_chunk_8.tre 0.0166339 15.2225 0.0689655172414
uce-120_chunk_6.tre 0.187599 4.6936 0.0689655172414
## [1] "The Robinson-Foulds distances of the three best chunks from the reference tree are as follows:"
##                     chunk  rf
## 1686 uce-625_chunk_10.tre 106
##                    chunk  rf
## 629 uce-1317_chunk_8.tre 111
##                   chunk  rf
## 429 uce-120_chunk_6.tre 114

The chunk with the minimum RF distance from the reference tree:

##                    chunk rf
## 678 uce-1340_chunk_2.tre 85

In order of descending priority: bipartition support, tree length, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-625_chunk_10.tre 0.0675117 0.706842 0.0775862068966
uce-120_chunk_6.tre 0.187599 4.6936 0.0689655172414
uce-1317_chunk_8.tre 0.0166339 15.2225 0.0689655172414

In order of descending priority: root-to-tip variance, tree length, bipartition support

name root-to-tip_var treelength bipartition
uce-1175_chunk_4.tre 0.0 8.48388e-07 0.00862068965517
uce-675_chunk_5.tre 0.0 8.58914e-07 0.0
uce-537_chunk_3.tre 0.0 8.71304e-07 0.00862068965517

In order of descending priority: root-to-tip variance, bipartition support, tree length

name root-to-tip_var treelength bipartition
uce-1189_chunk_2.tre 0.0 1.52836 0.0431034482759
uce-739_chunk_2.tre 0.0 2.2254e-06 0.0344827586207
uce-67_chunk_4.tre 0.0 0.0214551 0.0344827586207

In order of descending priority: tree length, root-to-tip variance, bipartition support

name root-to-tip_var treelength bipartition
uce-1175_chunk_4.tre 0.0 8.48388e-07 0.00862068965517
uce-675_chunk_5.tre 0.0 8.58914e-07 0.0
uce-537_chunk_3.tre 0.0 8.71304e-07 0.00862068965517

In order of descending priority: tree length, bipartition support, root-to-tip variance

name root-to-tip_var treelength bipartition
uce-1175_chunk_4.tre 0.0 8.48388e-07 0.00862068965517
uce-675_chunk_5.tre 0.0 8.58914e-07 0.0
uce-537_chunk_3.tre 0.0 8.71304e-07 0.00862068965517

Step 1 of filtering: remove trees of zero length

No effect: the minimum observed tree length is non-zero and equal to:

## [1] 8.48388e-07
## Number of trees after filtering: 2319
## Percentage of trees after filtering: 100%

Step 2 of filtering: remove the shortest and longest 10% of trees:

## Original min. tree length: 8.48388e-07
## Original max. tree length: 141.225
## Min. tree length after step 2: 0.111172
## Max. tree length after step 2: 20.8823
## Number of trees after step 2: 1855
## Percentage of trees after step 2: 79.99138%

Step 3 of filtering: remove the most unclocklike third of loci

## Max. root-to-tip variance from step 2: 40.1753
## Max. root-to-tip variance after step 3: 0.00177898
## Number of trees after step 3: 1243
## Percentage of trees after step 3: 53.60069%

Step 4 of filtering: keep the loci in the upper 10% of bipartition support

## Min. bipartition support from step 3: 0
## Min. bipartition support after step 4: 0.03448276
## Number of trees after step 4: 66
## Percentage of trees after step 4: 2.846054%

Out of the 66 chunks selected from the SH-like > 0.9 dataset, 65 were 50-bp long. To facilitate PartitionFinder searches, these chunks were aligned first using SequenceMatrix v1.8, with the last (47-bp) chunk attached to the resulting alignment afterwards. This made it easier to automate the calculation of base pair ranges for each locus (the latter needed to be included in the PartitionFinder configuration file). The outgroups (Alepisaurus ferox and Ceratoscopelus warmingii) were excluded from the concatenated alignment, as they were not present in the available topology constraint.

PartitionFinder was first run with the “rcluster” search option. Although the “models” option was set to “all”, since rcluster searches can only be performed in RAxML, this effectively limited the analysis to the three models implemented in the latter software (GTR, GTR+\(\Gamma\), GTR+\(\Gamma\)+I). This run yielded 8 subsets of the following properties:

Subset Best Model Sites Rate under GTR+\(\Gamma\) ID
1 GTR+\(\Gamma\) 1047 1.048826 76dec7513c9b37738bfac30d5d512cf3
2 GTR+I+\(\Gamma\) 1100 0.672610 111ec939b8c23b0e4e2529cee50f387a
3 GTR+\(\Gamma\) 550 0.281570 00fea7ffef92b99d78df6650b56ebb0c
4 GTR+\(\Gamma\) 100 0.965800 065a20f09eb1fa7383dedb55df51e100
5 GTR+\(\Gamma\) 200 2.278670 a6afbf36fc27e68c089e09c31ca3c71f
6 GTR+\(\Gamma\) 50 1.682760 da54ec6f6966e6b696433ef124df9e1f
7 GTR+I+\(\Gamma\) 50 2.739838 2d0d981211d9b6e3f364c1fcd221b555
8 GTR+\(\Gamma\) 200 1.356237 fd9894d33a357dcdab10c55c5153eb16

In the next step, the alignment was analyzed under the “greedy” search option, with the BEAST model set and without the “–raxml” flag. This search recovered the following 14 partitions:

Subset Best Model Sites Tree size under the best model ID
1 HKY+\(\Gamma\)+X 100 6.07552 3f353939a600a6973a6a1e0608925da6
2 TRNEF+\(\Gamma\) 200 4.34326 ceefdb63d1de1122404ab11285f05f27
3 JC+\(\Gamma\) 50 2.80253 ac2a21040350ec54b4664bc752a13a45
4 TRNEF+\(\Gamma\) 300 1.53147 baed377a41689244a752bc9e3169c975
5 GTR+I+\(\Gamma\)+X 350 3.44224 ce05da9c4ae84b4a74cb468b18fcda82
6 K80+I+\(\Gamma\) 450 3.71596 16b9f71868c90950d0dd144fd5de511f
7 SYM+\(\Gamma\) 300 8.15389 c5076d62640bd9eeabe53de6e5bf7f7a
8 K80+\(\Gamma\) 150 9.20524 d5aeca1d1fad19fc89dbb336bad98cbd
9 K80+\(\Gamma\) 497 5.13898 e7f992b256534832ccdb46c09aa6e9a4
10 HKY+\(\Gamma\)+X 300 2.87180 4e722846245f8cd6600c32f3131120a6
11 TRNEF+\(\Gamma\) 250 5.87708 72ad9849e1f3be2cbffa1097935e00fc
12 GTR+I+\(\Gamma\)+X 50 36.99851 2d0d981211d9b6e3f364c1fcd221b555
13 JC+\(\Gamma\) 150 1.20339 d687ea29d8ab88ed27334eafc8164ec8
14 TRNEF+\(\Gamma\) 150 13.44953 00f862e3cabdf1be1b99bc5b1455ba78

(In PhyML, “tree size” denotes the sum of edge lengths: see http://github.com/stephaneguindon/phyml/blob/master/doc/phyml-manual.tex. “X” denotes estimated, as opposed to empirical, base frequencies.)

BEAST analysis

The two 50-bp partitions were removed. Each partition received its own substitution model, which was generally identical to that selected by PartitionFinder, except for those models that contained the I parameter (proportion of invariant sites). Since this parameter and \(\Gamma\) account for the same phenomenon (rate heterogeneity across sites), their simultaneous inclusion causes the resulting model to be non-identifiable, leading to potential mixing problems in MCMC simulations. The models that were not directly available in BEAUTi were implemented as follows:

  • TrNef: TN93 + equal base frequencies
  • K80: HKY85 + equal base frequencies
  • SYM: GTR + equal base frequencies

Each \(\Gamma\) rate heterogeneity distribution was discretized into 4 categories. The default improper prior on the relative rates (the allMus parameter) was set to a gamma distribution with a shape of 0.001 and a scale of 1000 following the recommendations at https://www.biostat.washington.edu/sites/default/files/modules//2016_SISMID_13_11.pdf. All the priors on substitution model parameters were kept at their default values.

In contrast to the substitution models, the parameters of the uncorrelated lognormal clock model were linked across partitions. The ucld.mean hyperparameter was assigned a lognormal hyperprior with a mean of 0.005 (in real space) and a log standard deviation of 1, with the initial value set equal to 0.005. A truncated exponential hyperprior with support (0, 1), a mean of 0.3, and an initial value of 0.1 was used for ucld.stdev.

The “fixed tree topology” operator mix was used (based on a user-supplied topology common to all partitions), with the tuning of the ucld.mean and ucld.stdev operators set to 0.9 (default value = 0.75) and their weight doubled (from 3.0 to 6.0). Default tuning values and weights were used for all the remaining operators.

The 12 internal calibrations were all implemented as exponentials, and the (Polymixia + Aphredoderus) received the corrected calibration whose 95th percentile was equal to 116.35 Ma. A truncated exponential distribution supported on (98 Ma, 143 Ma) was constructed to calibrate the root of the tree, with its mean equal to the midpoint of the support interval (22.5 without the offset).

Finally, the MCMC simulation was run for 500 million generations, with a sampling period of 1000 generations.

beast -threads 12 -beagle_SSE concatchunks.xml

The analysis finished after 12.597 days and the resulting log file was examined using LogAnalyser to determine the EES value of each parameter:

java -Xmx6000m -cp /home/analysis/.linuxbrew/lib/beast.jar dr.app.tools.LogAnalyser -burnin 50000000 -ess concatchunks.log
##                        statistic          median         ESS
## 1                      posterior -48625.69180000   3413.0600
## 2                          prior  -2049.74610000    844.2895
## 3                     likelihood -46575.83370000  32241.1992
## 4           treeModel.rootHeight    137.72570000   2238.3418
## 5              tmrca(Aipichthys)    120.61650000   1324.2513
## 6           tmrca(Berybolcensis)     55.65810000   6036.6497
## 7               tmrca(Calatomus)     23.59360000    718.9452
## 8          tmrca(Chaetodontidae)     37.91700000   6567.4848
## 9           tmrca(Eastmanalepes)     50.96060000  16986.7366
## 10            tmrca(Eobuglossus)     43.01970000   8226.7418
## 11            tmrca(Eucoelopoma)     56.77560000   5699.0608
## 12         tmrca(Homonotichthys)     98.71850000   1207.3403
## 13           tmrca(Mcconichthys)     66.97340000    688.1744
## 14                   tmrca(Mene)     59.19900000   4229.6624
## 15         tmrca(Ramphexocoetus)     53.71620000   3420.8121
## 16                   tmrca(Root)    137.72570000   2238.3418
## 17                 tmrca(Tarkus)     49.83070000  13887.1125
## 18     birthDeath.meanGrowthRate      0.01870000   3785.5560
## 19  birthDeath.relativeDeathRate      0.05030000 125120.0000
## 20            Subset1HKYGX.kappa      3.30330000 336740.0000
## 21     Subset1HKYGX.frequencies1      0.38220000 160000.0000
## 22     Subset1HKYGX.frequencies2      0.23020000 194740.0000
## 23     Subset1HKYGX.frequencies3      0.15200000 188220.0000
## 24     Subset1HKYGX.frequencies4      0.23390000 191870.0000
## 25            Subset1HKYGX.alpha      0.98970000 344070.0000
## 26          Subset2TRNEFG.kappa1      4.82390000 339050.0000
## 27          Subset2TRNEFG.kappa2      1.81510000 353590.0000
## 28           Subset2TRNEFG.alpha      0.67370000 343600.0000
## 29          Subset3TRNEFG.kappa1      3.05060000 330620.0000
## 30          Subset3TRNEFG.kappa2      4.86070000 327930.0000
## 31           Subset3TRNEFG.alpha      0.46710000 335870.0000
## 32              Subset4GTRIGX.ac      0.17860000 142440.0000
## 33              Subset4GTRIGX.ag      0.41800000  69343.9763
## 34              Subset4GTRIGX.at      0.10600000 158030.0000
## 35              Subset4GTRIGX.cg      0.34480000 106840.0000
## 36              Subset4GTRIGX.gt      0.14930000 132440.0000
## 37    Subset4GTRIGX.frequencies1      0.30180000 109180.0000
## 38    Subset4GTRIGX.frequencies2      0.21230000  87820.5730
## 39    Subset4GTRIGX.frequencies3      0.31520000  91691.2293
## 40    Subset4GTRIGX.frequencies4      0.16960000  98336.1071
## 41           Subset4GTRIGX.alpha      0.44590000 276290.0000
## 42            Subset5K80IG.kappa      3.56400000 364980.0000
## 43            Subset5K80IG.alpha      0.31900000 370210.0000
## 44                Subset6SYMG.ac      0.31450000 233040.0000
## 45                Subset6SYMG.ag      1.61400000 171440.0000
## 46                Subset6SYMG.at      0.24760000 241230.0000
## 47                Subset6SYMG.cg      1.23840000 197650.0000
## 48                Subset6SYMG.gt      0.24490000 285190.0000
## 49             Subset6SYMG.alpha      0.52120000 361230.0000
## 50             Subset7K80G.kappa      2.93730000 365130.0000
## 51             Subset7K80G.alpha      0.70830000 361470.0000
## 52             Subset8K80G.kappa      3.17480000 373600.0000
## 53             Subset8K80G.alpha      1.22040000 364000.0000
## 54            Subset9HKYGX.kappa      3.59770000 358280.0000
## 55     Subset9HKYGX.frequencies1      0.20910000 185070.0000
## 56     Subset9HKYGX.frequencies2      0.25900000 189260.0000
## 57     Subset9HKYGX.frequencies3      0.30580000 174970.0000
## 58     Subset9HKYGX.frequencies4      0.22510000 192860.0000
## 59            Subset9HKYGX.alpha      0.62780000 344070.0000
## 60         Subset10TRNEFG.kappa1      4.17550000 337400.0000
## 61         Subset10TRNEFG.kappa2      6.02560000 342470.0000
## 62          Subset10TRNEFG.alpha      0.50600000 338290.0000
## 63             Subset11JCG.alpha      0.38770000 282140.0000
## 64         Subset12TRNEFG.kappa1      3.08980000 324920.0000
## 65         Subset12TRNEFG.kappa2      7.49850000 321000.0000
## 66          Subset12TRNEFG.alpha      0.33910000 220060.0000
## 67               Subset1HKYGX.mu      1.17860000 183520.0000
## 68              Subset2TRNEFG.mu      0.86830000 155210.0000
## 69              Subset3TRNEFG.mu      0.28780000 112480.0000
## 70              Subset4GTRIGX.mu      0.71110000 105770.0000
## 71               Subset5K80IG.mu      0.81240000  89865.7064
## 72                Subset6SYMG.mu      1.65800000  86079.4311
## 73                Subset7K80G.mu      1.92930000 131010.0000
## 74                Subset8K80G.mu      1.05670000  92400.6821
## 75               Subset9HKYGX.mu      0.59180000 131650.0000
## 76             Subset10TRNEFG.mu      1.18620000 132450.0000
## 77                Subset11JCG.mu      0.20350000 110950.0000
## 78             Subset12TRNEFG.mu      2.54390000  79083.6503
## 79                     ucld.mean      0.00073594    332.5618
## 80                    ucld.stdev      0.98430000   2649.2935
## 81                      meanRate      0.00068979    700.5540
## 82        coefficientOfVariation      1.27000000    226.7723
## 83                    covariance      0.23290000    433.6862
## 84   Subset1HKYGX.treeLikelihood  -1890.61200000  36770.2075
## 85  Subset2TRNEFG.treeLikelihood  -2931.73400000  35236.5308
## 86  Subset3TRNEFG.treeLikelihood  -2228.42960000  36860.1607
## 87  Subset4GTRIGX.treeLikelihood  -4743.64810000  29583.0309
## 88   Subset5K80IG.treeLikelihood  -5280.81190000  32435.6649
## 89    Subset6SYMG.treeLikelihood  -5733.31740000  32350.1559
## 90    Subset7K80G.treeLikelihood  -3537.00090000  26073.9083
## 91    Subset8K80G.treeLikelihood  -8747.29310000  18666.6737
## 92   Subset9HKYGX.treeLikelihood  -3646.88110000  24260.8424
## 93 Subset10TRNEFG.treeLikelihood  -4255.05270000  39038.3017
## 94    Subset11JCG.treeLikelihood   -949.70740000   9002.7411
## 95 Subset12TRNEFG.treeLikelihood  -2630.61690000  25585.5111
## 96                   branchRates      0.00000000   1000.0000
## 97                    speciation   -580.10770000    566.5915

Since all the ESS values exceeded 200, the sample from the posterior was summarized using TreeAnnotator:

java -Xmx20000m -cp /home/analysis/.linuxbrew/lib/beast.jar dr.app.tools.TreeAnnotator -burnin 50000000 -heights median concatchunks.trees chunks-BEAST.tre
Comparison with the MCMCTree-generated tree from the manuscript

Comparison with the MCMCTree-generated tree from the manuscript

BEAST: Random local clocks

Two more analyses were performed using BEAST; their settings were identical to those described above except for the clock model used. In both cases, a random local clock model was utilized, with one analysis using the uncorrelated version of the model (with the ratesAreMultipliers option in the XML file set to the default value of "false") and the other employing the correlated version (ratesAreMultipliers="true"). In both analyses, default priors were placed on the parameters of the clock model.

beast -threads 6 -beagle_SSE concatchunks-urlc.xml
beast -threads 6 -beagle_SSE concatchunks-crlc.xml

(Note: these chains did not reach the target length due to a power outage.)

BEAST: Extended calibration set

Another analysis was started with settings identical to the first one (in particular, with the uncorrelated lognormal relaxed clock model), but with the extended dataset of 28 calibrations:

/home/linuxbrew/.linuxbrew/bin/beast -beagle_SSE -beagle_scaling dynamic concatchunks-ext.xml

This run finished after 7.059 days, with the following effective sample sizes for individual parameters (as determined by LogAnalyser):

/home/linuxbrew/.linuxbrew/bin/loganalyser -burnin 50000000 -ess concatchunks-ext.log
##                         statistic          median        ESS
## 1                       posterior -48684.38740000  2763.4231
## 2                           prior  -2109.18830000   721.1500
## 3                      likelihood -46575.16520000 13844.4609
## 4            treeModel.rootHeight    136.14570000  1743.5335
## 5               tmrca(Aipichthys)    122.76580000  1222.0035
## 6         tmrca(Archaeotetraodon)     36.63320000   198.8701
## 7                 tmrca(Archaeus)     56.60770000  8352.3874
## 8            tmrca(Berybolcensis)     54.67900000  7254.2808
## 9                tmrca(Calatomus)     26.00620000   399.8151
## 10          tmrca(Chaetodontidae)     41.24440000  6296.0075
## 11                tmrca(Cretzeus)     94.24210000   135.2974
## 12           tmrca(Eastmanalepes)     52.64200000 14885.0059
## 13                tmrca(Eobothus)     57.80940000  1785.4873
## 14             tmrca(Eobuglossus)     43.29610000  4816.3060
## 15             tmrca(Eocoelopoma)     56.88220000  5205.1277
## 16       tmrca(Gasterorhamphosus)     75.07320000  2132.2268
## 17          tmrca(Homonotichthys)     99.30460000  1329.7146
## 18             tmrca(Hoplopteryx)    127.44200000  1086.9194
## 19                 tmrca(Luvarus)     65.22740000   809.0498
## 20          tmrca(Mahengechromis)     48.20490000  7659.4325
## 21         tmrca(Massamorichthys)     63.21960000   600.5667
## 22                    tmrca(Mene)     58.76210000  4622.4281
## 23           tmrca(Proacanthurus)     51.23210000  6784.3879
## 24         tmrca(Prosolenostomus)     59.10850000   821.7280
## 25         tmrca(Rhamphexocoetus)     54.45010000  6244.0991
## 26           tmrca(Rhinocephalus)     56.57650000  2295.8754
## 27                    tmrca(Root)    136.14570000  1743.5335
## 28               tmrca(Sphyraena)     59.69350000  8346.0325
## 29                  tmrca(Tarkus)     49.83670000  9320.9490
## 30            tmrca(Trachipterus)     25.06320000   870.6285
## 31                 tmrca(Triodon)     53.63450000   612.9674
## 32                tmrca(Turkmene)     64.00440000   855.3938
## 33                tmrca(Zenopsis)     37.80840000   420.6294
## 34      birthDeath.meanGrowthRate      0.01850000  3892.3232
## 35   birthDeath.relativeDeathRate      0.04550000 15134.3519
## 36             Subset1HKYGX.kappa      3.30200000 18001.0000
## 37      Subset1HKYGX.frequencies1      0.38230000 18001.0000
## 38      Subset1HKYGX.frequencies2      0.23000000 18001.0000
## 39      Subset1HKYGX.frequencies3      0.15230000 18001.0000
## 40      Subset1HKYGX.frequencies4      0.23400000 18001.0000
## 41             Subset1HKYGX.alpha      0.98980000 17491.3422
## 42           Subset2TRNEFG.kappa1      4.81920000 18001.0000
## 43           Subset2TRNEFG.kappa2      1.81260000 17951.5243
## 44            Subset2TRNEFG.alpha      0.67220000 17988.8690
## 45           Subset3TRNEFG.kappa1      3.04990000 17514.5608
## 46           Subset3TRNEFG.kappa2      4.84920000 18001.0000
## 47            Subset3TRNEFG.alpha      0.46630000 17907.2312
## 48               Subset4GTRIGX.ac      0.17760000 18001.0000
## 49               Subset4GTRIGX.ag      0.41520000 17530.1025
## 50               Subset4GTRIGX.at      0.10570000 18001.0000
## 51               Subset4GTRIGX.cg      0.34220000 17165.8646
## 52               Subset4GTRIGX.gt      0.14880000 17568.1246
## 53     Subset4GTRIGX.frequencies1      0.30190000 17366.2591
## 54     Subset4GTRIGX.frequencies2      0.21200000 17969.0065
## 55     Subset4GTRIGX.frequencies3      0.31540000 17187.6766
## 56     Subset4GTRIGX.frequencies4      0.16930000 17966.0438
## 57            Subset4GTRIGX.alpha      0.44540000 18001.0000
## 58             Subset5K80IG.kappa      3.56170000 18001.0000
## 59             Subset5K80IG.alpha      0.31900000 18001.0000
## 60                 Subset6SYMG.ac      0.31350000 16789.4178
## 61                 Subset6SYMG.ag      1.60860000 18001.0000
## 62                 Subset6SYMG.at      0.24720000 17544.8122
## 63                 Subset6SYMG.cg      1.23730000 17490.5724
## 64                 Subset6SYMG.gt      0.24400000 18001.0000
## 65              Subset6SYMG.alpha      0.51970000 18001.0000
## 66              Subset7K80G.kappa      2.93720000 18001.0000
## 67              Subset7K80G.alpha      0.70890000 18001.0000
## 68              Subset8K80G.kappa      3.17600000 17757.1075
## 69              Subset8K80G.alpha      1.21870000 17392.0732
## 70             Subset9HKYGX.kappa      3.59600000 17229.8960
## 71      Subset9HKYGX.frequencies1      0.20940000 17411.0904
## 72      Subset9HKYGX.frequencies2      0.25880000 18001.0000
## 73      Subset9HKYGX.frequencies3      0.30590000 17833.6630
## 74      Subset9HKYGX.frequencies4      0.22510000 17823.9222
## 75             Subset9HKYGX.alpha      0.62670000 17827.3347
## 76          Subset10TRNEFG.kappa1      4.17290000 17254.1945
## 77          Subset10TRNEFG.kappa2      6.01360000 18001.0000
## 78           Subset10TRNEFG.alpha      0.50490000 17516.4006
## 79              Subset11JCG.alpha      0.38730000 18001.0000
## 80          Subset12TRNEFG.kappa1      3.08330000 17424.3666
## 81          Subset12TRNEFG.kappa2      7.49460000 18001.0000
## 82           Subset12TRNEFG.alpha      0.33860000 17339.1346
## 83                Subset1HKYGX.mu      1.18330000 17737.5069
## 84               Subset2TRNEFG.mu      0.86810000 17265.7801
## 85               Subset3TRNEFG.mu      0.28770000 18001.0000
## 86               Subset4GTRIGX.mu      0.71030000 17282.8179
## 87                Subset5K80IG.mu      0.81110000 17571.3579
## 88                 Subset6SYMG.mu      1.65750000 18001.0000
## 89                 Subset7K80G.mu      1.93380000 18001.0000
## 90                 Subset8K80G.mu      1.05690000 16988.5077
## 91                Subset9HKYGX.mu      0.59210000 18001.0000
## 92              Subset10TRNEFG.mu      1.18530000 17556.4438
## 93                 Subset11JCG.mu      0.20370000 18001.0000
## 94              Subset12TRNEFG.mu      2.55030000 16378.8994
## 95                      ucld.mean      0.00072704   197.9636
## 96                     ucld.stdev      0.98360000   866.6601
## 97                       meanRate      0.00068166   771.9061
## 98         coefficientOfVariation      1.29570000   143.9961
## 99                     covariance      0.23880000   208.4930
## 100   Subset1HKYGX.treeLikelihood  -1890.75370000 15350.8909
## 101  Subset2TRNEFG.treeLikelihood  -2931.99210000 15300.1650
## 102  Subset3TRNEFG.treeLikelihood  -2228.20130000 15363.0960
## 103  Subset4GTRIGX.treeLikelihood  -4743.39680000 12599.6925
## 104   Subset5K80IG.treeLikelihood  -5281.05330000 16367.2093
## 105    Subset6SYMG.treeLikelihood  -5733.17950000 13937.6508
## 106    Subset7K80G.treeLikelihood  -3536.68580000 11820.6143
## 107    Subset8K80G.treeLikelihood  -8747.09380000 13462.9728
## 108   Subset9HKYGX.treeLikelihood  -3646.62540000 13764.4572
## 109 Subset10TRNEFG.treeLikelihood  -4254.89780000 12708.1486
## 110    Subset11JCG.treeLikelihood   -949.69550000  8979.5379
## 111 Subset12TRNEFG.treeLikelihood  -2630.68180000 15615.8354
## 112                    speciation   -581.46880000   546.6950

Since four parameters had ESS values of less than 200 (tmrca(Archaeotetraodon), tmrca(Cretzeus), ucld.mean, coefficientOfVariation), a new run was started with identical settings and the chain length set to 250 million generations:

/home/linuxbrew/.linuxbrew/bin/beast -beagle_SSE -beagle_scaling dynamic concatchunks-ext2.xml

The second run finished after 4.384 days.

The next steps were as follows:

  1. Remove the first 10% of samples as burnin from both chains to produce truncated log files:

    /home/linuxbrew/.linuxbrew/bin/logcombiner -burnin 50000001 concatchunks-ext.log concatchunks-ext-no-burnin.log
    /home/linuxbrew/.linuxbrew/bin/logcombiner -burnin 25000001 concatchunks-ext2.log concatchunks-ext2-no-burnin.log
  2. Combine the truncated log files:

    /home/linuxbrew/.linuxbrew/bin/logcombiner -burnin 0 concatchunks-ext-no-burnin.log concatchunks-ext2-no-burnin.log concatchunks-ext-combined.log
  3. Analyze the combined log file to assess ESS values:

    /home/linuxbrew/.linuxbrew/bin/loganalyser -burnin 0 -ess concatchunks-ext-combined.log

Although the ESS values for the 4 previously problematic parameters all exceeded 200 when the two runs were combined, the ESS value of one more parameter (tmrca(Tarkus))unexpectedly dropped below the threshold:

##                         statistic          median        ESS
## 1                       posterior -48684.61890000  4018.7952
## 2                           prior  -2109.34860000  1006.3031
## 3                      likelihood -46575.17930000 21777.6112
## 4            treeModel.rootHeight    136.07150000  2642.6995
## 5               tmrca(Aipichthys)    122.59890000  2219.8782
## 6         tmrca(Archaeotetraodon)     36.68160000   300.9514
## 7                 tmrca(Archaeus)     56.60900000 12713.2916
## 8            tmrca(Berybolcensis)     54.64290000 11218.4625
## 9                tmrca(Calatomus)     25.72650000   613.1224
## 10          tmrca(Chaetodontidae)     41.25430000  8190.6974
## 11                tmrca(Cretzeus)     93.90460000   237.4754
## 12           tmrca(Eastmanalepes)     52.64260000 22170.2330
## 13                tmrca(Eobothus)     57.91220000  2680.7794
## 14             tmrca(Eobuglossus)     43.31750000  7470.8790
## 15             tmrca(Eocoelopoma)     56.87570000  8276.2164
## 16       tmrca(Gasterorhamphosus)     75.05650000  3230.4774
## 17          tmrca(Homonotichthys)     99.22490000  2555.9326
## 18             tmrca(Hoplopteryx)    127.20650000  1478.2580
## 19                 tmrca(Luvarus)     65.31620000  1396.4432
## 20          tmrca(Mahengechromis)     48.17900000 11840.4841
## 21         tmrca(Massamorichthys)     63.33570000  1009.8491
## 22                    tmrca(Mene)     58.82670000  6847.9146
## 23           tmrca(Proacanthurus)     51.25330000 10184.5107
## 24         tmrca(Prosolenostomus)     59.06560000  1226.1886
## 25         tmrca(Rhamphexocoetus)     54.36880000  8583.6205
## 26           tmrca(Rhinocephalus)     56.60650000  3439.4406
## 27                    tmrca(Root)    136.07150000  2642.6995
## 28               tmrca(Sphyraena)     59.76390000 12653.9557
## 29                  tmrca(Tarkus)     49.98980000   126.1217
## 30            tmrca(Trachipterus)     25.29110000  1336.7531
## 31                 tmrca(Triodon)     53.71510000   937.0322
## 32                tmrca(Turkmene)     64.05950000  1416.0132
## 33                tmrca(Zenopsis)     37.73790000   802.1615
## 34      birthDeath.meanGrowthRate      0.01850000  5668.8584
## 35   birthDeath.relativeDeathRate      0.04530000 23256.0832
## 36             Subset1HKYGX.kappa      3.30360000 26678.2412
## 37      Subset1HKYGX.frequencies1      0.38220000 27000.0000
## 38      Subset1HKYGX.frequencies2      0.23020000 27000.0000
## 39      Subset1HKYGX.frequencies3      0.15220000 27000.0000
## 40      Subset1HKYGX.frequencies4      0.23400000 27000.0000
## 41             Subset1HKYGX.alpha      0.98870000 26584.5037
## 42           Subset2TRNEFG.kappa1      4.82280000 27000.0000
## 43           Subset2TRNEFG.kappa2      1.81350000 26752.9895
## 44            Subset2TRNEFG.alpha      0.67290000 26917.0274
## 45           Subset3TRNEFG.kappa1      3.05000000 27000.0000
## 46           Subset3TRNEFG.kappa2      4.84750000 27000.0000
## 47            Subset3TRNEFG.alpha      0.46620000 27000.0000
## 48               Subset4GTRIGX.ac      0.17750000 27000.0000
## 49               Subset4GTRIGX.ag      0.41500000 25570.6224
## 50               Subset4GTRIGX.at      0.10570000 27000.0000
## 51               Subset4GTRIGX.cg      0.34230000 26434.8037
## 52               Subset4GTRIGX.gt      0.14860000 26976.0510
## 53     Subset4GTRIGX.frequencies1      0.30190000 26472.4252
## 54     Subset4GTRIGX.frequencies2      0.21200000 26641.8780
## 55     Subset4GTRIGX.frequencies3      0.31550000 26599.0962
## 56     Subset4GTRIGX.frequencies4      0.16930000 26366.7657
## 57            Subset4GTRIGX.alpha      0.44520000 27000.0000
## 58             Subset5K80IG.kappa      3.55900000 27000.0000
## 59             Subset5K80IG.alpha      0.31910000 26404.5070
## 60                 Subset6SYMG.ac      0.31370000 25467.5781
## 61                 Subset6SYMG.ag      1.60920000 27000.0000
## 62                 Subset6SYMG.at      0.24730000 27000.0000
## 63                 Subset6SYMG.cg      1.23790000 26676.0355
## 64                 Subset6SYMG.gt      0.24420000 27000.0000
## 65              Subset6SYMG.alpha      0.52000000 26757.9001
## 66              Subset7K80G.kappa      2.93550000 27000.0000
## 67              Subset7K80G.alpha      0.70880000 27000.0000
## 68              Subset8K80G.kappa      3.17590000 27000.0000
## 69              Subset8K80G.alpha      1.22000000 26734.3603
## 70             Subset9HKYGX.kappa      3.59430000 26262.9651
## 71      Subset9HKYGX.frequencies1      0.20920000 26249.4586
## 72      Subset9HKYGX.frequencies2      0.25890000 26904.3496
## 73      Subset9HKYGX.frequencies3      0.30580000 26158.2443
## 74      Subset9HKYGX.frequencies4      0.22510000 26447.5814
## 75             Subset9HKYGX.alpha      0.62670000 26428.7697
## 76          Subset10TRNEFG.kappa1      4.17090000 26416.2617
## 77          Subset10TRNEFG.kappa2      6.01590000 27000.0000
## 78           Subset10TRNEFG.alpha      0.50510000 26076.5998
## 79              Subset11JCG.alpha      0.38700000 27000.0000
## 80          Subset12TRNEFG.kappa1      3.08690000 26675.3958
## 81          Subset12TRNEFG.kappa2      7.50120000 27000.0000
## 82           Subset12TRNEFG.alpha      0.33850000 27000.0000
## 83                Subset1HKYGX.mu      1.18280000 26828.6938
## 84               Subset2TRNEFG.mu      0.86800000 26255.6956
## 85               Subset3TRNEFG.mu      0.28790000 27000.0000
## 86               Subset4GTRIGX.mu      0.71010000 25129.2484
## 87                Subset5K80IG.mu      0.81080000 26461.5267
## 88                 Subset6SYMG.mu      1.65730000 27000.0000
## 89                 Subset7K80G.mu      1.93370000 26130.1157
## 90                 Subset8K80G.mu      1.05650000 27000.0000
## 91                Subset9HKYGX.mu      0.59240000 25762.3514
## 92              Subset10TRNEFG.mu      1.18560000 26783.9906
## 93                 Subset11JCG.mu      0.20370000 27000.0000
## 94              Subset12TRNEFG.mu      2.55020000 24805.2309
## 95                      ucld.mean      0.00072680   294.7377
## 96                     ucld.stdev      0.98350000  1554.3464
## 97                       meanRate      0.00068161   995.7414
## 98         coefficientOfVariation      1.29670000   231.1065
## 99                     covariance      0.24070000   453.1176
## 100   Subset1HKYGX.treeLikelihood  -1890.75160000 22436.7525
## 101  Subset2TRNEFG.treeLikelihood  -2932.00810000 23328.1981
## 102  Subset3TRNEFG.treeLikelihood  -2228.22220000 23305.2617
## 103  Subset4GTRIGX.treeLikelihood  -4743.39800000 18857.7749
## 104   Subset5K80IG.treeLikelihood  -5281.08270000 22518.3527
## 105    Subset6SYMG.treeLikelihood  -5733.24140000 21927.9543
## 106    Subset7K80G.treeLikelihood  -3536.71940000 17624.6335
## 107    Subset8K80G.treeLikelihood  -8746.96880000 16840.0116
## 108   Subset9HKYGX.treeLikelihood  -3646.70940000 21631.2317
## 109 Subset10TRNEFG.treeLikelihood  -4254.92840000 19433.2103
## 110    Subset11JCG.treeLikelihood   -949.68370000 14102.6829
## 111 Subset12TRNEFG.treeLikelihood  -2630.64210000 19591.9161
## 112                   branchRates      0.00000000 25000.0000
## 113                    speciation   -581.49490000   733.9149

To assess whether this reflects a more general problem with convergence, maximum clade credibility trees were constructed for both the first run alone (18,000 samples excluding burnin) and the combined run (27,000 samples excluding burnin) and compared to each other:

/home/linuxbrew/.linuxbrew/bin/treeannotator -burnin 50000000 -heights median concatchunks-ext.trees chunks-ext-run1.tre
/home/linuxbrew/.linuxbrew/bin/logcombiner -trees -burnin 50000001 concatchunks-ext.trees concatchunks-ext-no-burnin.trees
/home/linuxbrew/.linuxbrew/bin/logcombiner -trees -burnin 25000001 concatchunks-ext2.trees concatchunks-ext2-no-burnin.trees
/home/linuxbrew/.linuxbrew/bin/logcombiner -trees -burnin 0 concatchunks-ext-no-burnin.trees concatchunks-ext2-no-burnin.trees concatchunks-ext-combined.trees
/home/linuxbrew/.linuxbrew/bin/treeannotator -burnin 0 -heights median concatchunks-ext-combined.trees chunks-ext-combined.tre

As can be seen, the ages are nearly identical:

BEAST under the extended calibration set: first run only vs. combined run

BEAST under the extended calibration set: first run only vs. combined run

Comparison of the BEAST analyses under the uncorrelated lognormal clock and 12/28 calibrations with the manuscript tree

Comparison of the BEAST analyses under the uncorrelated lognormal clock and 12/28 calibrations with the manuscript tree

BEAST: Fixed local clocks

In the next step, a number of nodes were assigned their own (strict) clocks:

  • Eupercaria (Canthigaster figueiredoi + Cephalopholis argus)
  • Carangaria (Centropomus medius + Alepes kleinii)
  • Ovalenteria (Xenentodon cancila + Emblemariopsis sp.)
  • Pelagiaria (Chiasmodon niger + Psenes cyanophrys)
  • Syngnathiformes (Syngnathus fuscus + Pseudupeneus maculatus)
  • Gobiaria (Kurtus gulliveri + Periophthalmus barbarus)
  • Lepophidium profundorum + Carapus bermudensis
  • Rondeletiidae + Holocentridae (Rondeletia loricata + Myripristis leiognathus)
  • Paracanthopterygii (Lampris guttatus + Gadus morhua)

The objective was to assign a separate clock to each of the six major percomorph subgroups (Eupercaria, Carangaria, Ovalentaria, Pelagiaria, Syngnathiformes, Gobiaria), and then to each of their sister groups that branched off successively closer to the root, ending with the Paracanthopterygii. However, as BEAST is unable to give local clocks to individual tips, only outgroups comprising two or more terminal taxa were assigned their own clocks, while the rates of Batrachoides and Anoplogaster were governed by a common tree-wide clock (set up by the clock.rate parameter) separate from all the local clocks. Both the local clocks as well as the tree-wide clock were strict, and all were assigned the same prior: a lognormal distribution with a mean of 0.005 (in real space), a standard deviation of 1, and an initial value of 0.005.

By default, BEAST assumes that if the fixed local clock model is selected, the subset of nodes that are to be assigned their own clocks is identical to the set of calibrated nodes. This was not the case in the analysis below. However, this problem can be solved in the following way: (1) In the “Taxa” tab of BEAUTi, enforce the monophyly of those nodes which are going to receive their own clocks but not the monophyly of the calibrated nodes. Note that this makes no difference in practice, since the analysis is to be run on a fixed topology, but it does prevent BEAST from assigning local clocks to the calibrated nodes. (2) In the “Priors” tab of BEAUTi, both the calibrated nodes and the “clock-bearing” nodes are expected to receive calibration priors. While these are only available for the former category, the latter category can be effectively freed from the need to specify priors by selecting the Use Tree Prior option in BEAUTi.

All the other settings (substitution models, calibration densities, the fixed topology operator mix) were identical to the uncorrelated lognormal clock analysis described above.

The analysis was run as follows:

/home/linuxbrew/.linuxbrew/bin/beast -beagle_SSE -beagle_scaling dynamic fixed-local-strict-13-calib.xml

The chain reached the target length of 500 million generations after 11.626 days.

The effective samples sizes of individual parameters as calculated by LogAnalyser were as follows:

/home/linuxbrew/.linuxbrew/bin/loganalyser -burnin 50000000 -ess fixed-local-strict-13-calib.log
##                              statistic          median        ESS
## 1                            posterior -49123.84490000  2037.3359
## 2                                prior   -730.66560000   371.6547
## 3                           likelihood -48392.96990000 11889.6246
## 4                 treeModel.rootHeight    142.38790000 14889.8482
## 5                    tmrca(Aipichthys)    141.47640000 16338.0549
## 6                 tmrca(Berybolcensis)     73.45960000 13030.8789
## 7                     tmrca(Calatomus)     33.12950000   864.5692
## 8                    tmrca(Carangaria)     69.71650000   405.1534
## 9           tmrca(Carapus-Lepophidium)     60.37370000 16298.4431
## 10               tmrca(Chaetodontidae)     31.51700000  6679.4499
## 11                tmrca(Eastmanalepes)     49.54820000 12415.8822
## 12                  tmrca(Eobuglossus)     50.93410000  2272.5621
## 13                  tmrca(Eucoelopoma)     55.98260000 10615.3727
## 14                   tmrca(Eupercaria)     71.27540000   142.9556
## 15                     tmrca(Gobiaria)     57.76250000   616.5514
## 16               tmrca(Homonotichthys)    125.84800000 16307.8877
## 17                 tmrca(Mcconichthys)     63.40650000  6719.6232
## 18                         tmrca(Mene)     58.17530000  5287.7434
## 19                  tmrca(Ovalentaria)     58.53990000   926.1633
## 20                   tmrca(Pelagiaria)     73.88320000   954.1835
## 21               tmrca(Ramphexocoetus)     54.27350000  1819.7195
## 22  tmrca(Rondeletiidae-Holocentridae)    105.59100000 11628.5200
## 23                         tmrca(Root)    142.38790000 14889.8482
## 24              tmrca(Syngnathiformes)     67.57600000  1080.1733
## 25                       tmrca(Tarkus)     49.70150000  1083.8428
## 26           tmrca(Paracanthopterygii)    141.47640000 16338.0549
## 27           birthDeath.meanGrowthRate      0.01940000  7300.9057
## 28        birthDeath.relativeDeathRate      0.05580000 17849.9035
## 29                  Subset1HKYGX.kappa      3.23410000 18001.0000
## 30           Subset1HKYGX.frequencies1      0.38410000 17771.2752
## 31           Subset1HKYGX.frequencies2      0.23270000 17483.9830
## 32           Subset1HKYGX.frequencies3      0.15110000 18001.0000
## 33           Subset1HKYGX.frequencies4      0.23090000 16701.7774
## 34                  Subset1HKYGX.alpha      1.04300000 18001.0000
## 35                Subset2TRNEFG.kappa1      4.70270000 16211.6753
## 36                Subset2TRNEFG.kappa2      1.75900000 17135.2751
## 37                 Subset2TRNEFG.alpha      0.68160000 18001.0000
## 38                Subset3TRNEFG.kappa1      3.02350000 17716.5092
## 39                Subset3TRNEFG.kappa2      4.82520000 16905.5261
## 40                 Subset3TRNEFG.alpha      0.47330000 17606.3865
## 41                    Subset4GTRIGX.ac      0.18730000 15377.2684
## 42                    Subset4GTRIGX.ag      0.42460000 17598.3592
## 43                    Subset4GTRIGX.at      0.10000000 16575.0251
## 44                    Subset4GTRIGX.cg      0.36950000 18001.0000
## 45                    Subset4GTRIGX.gt      0.14950000 18001.0000
## 46          Subset4GTRIGX.frequencies1      0.30510000 17825.4731
## 47          Subset4GTRIGX.frequencies2      0.20550000 16197.4361
## 48          Subset4GTRIGX.frequencies3      0.31080000 16847.2357
## 49          Subset4GTRIGX.frequencies4      0.17760000 18001.0000
## 50                 Subset4GTRIGX.alpha      0.45330000 18001.0000
## 51                  Subset5K80IG.kappa      3.52390000 18001.0000
## 52                  Subset5K80IG.alpha      0.32360000 18001.0000
## 53                      Subset6SYMG.ac      0.33140000 17418.7030
## 54                      Subset6SYMG.ag      1.57810000 18001.0000
## 55                      Subset6SYMG.at      0.24350000 18001.0000
## 56                      Subset6SYMG.cg      1.25130000 17269.7941
## 57                      Subset6SYMG.gt      0.26090000 18001.0000
## 58                   Subset6SYMG.alpha      0.52430000 17327.8499
## 59                   Subset7K80G.kappa      2.84290000 16893.2805
## 60                   Subset7K80G.alpha      0.74240000 16946.4298
## 61                   Subset8K80G.kappa      3.13520000 18001.0000
## 62                   Subset8K80G.alpha      1.25100000 16472.9896
## 63                  Subset9HKYGX.kappa      3.53970000 18001.0000
## 64           Subset9HKYGX.frequencies1      0.21280000 18001.0000
## 65           Subset9HKYGX.frequencies2      0.25950000 18001.0000
## 66           Subset9HKYGX.frequencies3      0.30070000 18001.0000
## 67           Subset9HKYGX.frequencies4      0.22630000 18001.0000
## 68                  Subset9HKYGX.alpha      0.64480000 18001.0000
## 69               Subset10TRNEFG.kappa1      4.13090000 18001.0000
## 70               Subset10TRNEFG.kappa2      5.85360000 18001.0000
## 71                Subset10TRNEFG.alpha      0.52020000 17821.6012
## 72                   Subset11JCG.alpha      0.38460000 18001.0000
## 73               Subset12TRNEFG.kappa1      3.02610000 18001.0000
## 74               Subset12TRNEFG.kappa2      6.89270000 18001.0000
## 75                Subset12TRNEFG.alpha      0.35850000 16840.6897
## 76                     Subset1HKYGX.mu      1.14530000 16457.0492
## 77                    Subset2TRNEFG.mu      0.87810000 18001.0000
## 78                    Subset3TRNEFG.mu      0.29580000 18001.0000
## 79                    Subset4GTRIGX.mu      0.73260000 18001.0000
## 80                     Subset5K80IG.mu      0.82460000 18001.0000
## 81                      Subset6SYMG.mu      1.69630000 18001.0000
## 82                      Subset7K80G.mu      1.85100000 18001.0000
## 83                      Subset8K80G.mu      1.06370000 18001.0000
## 84                     Subset9HKYGX.mu      0.59430000 17868.0226
## 85                   Subset10TRNEFG.mu      1.18480000 18001.0000
## 86                      Subset11JCG.mu      0.21330000 18001.0000
## 87                   Subset12TRNEFG.mu      2.42210000 18001.0000
## 88                          clock.rate      0.00059583  6499.0917
## 89                     Carangaria.rate      0.00038146  1707.5095
## 90            Carapus-Lepophidium.rate      0.00296050 17208.2276
## 91                     Eupercaria.rate      0.00063338   250.5997
## 92                       Gobiaria.rate      0.00177240   589.3144
## 93                    Ovalentaria.rate      0.00051243  1231.6271
## 94                     Pelagiaria.rate      0.00018009  2855.7223
## 95    Rondeletiidae-Holocentridae.rate      0.00016488 15539.6245
## 96                Syngnathiformes.rate      0.00146150  1436.4311
## 97             Paracanthopterygii.rate      0.00101200 16619.8225
## 98                            meanRate      0.00068337   633.3408
## 99              coefficientOfVariation      0.62670000   633.9622
## 100                         covariance      0.91110000  1054.6547
## 101        Subset1HKYGX.treeLikelihood  -1961.68430000 11412.9578
## 102       Subset2TRNEFG.treeLikelihood  -3050.01030000 15769.5321
## 103       Subset3TRNEFG.treeLikelihood  -2312.82940000  8179.2027
## 104       Subset4GTRIGX.treeLikelihood  -4909.92280000 12746.3787
## 105        Subset5K80IG.treeLikelihood  -5428.25720000 10132.1210
## 106         Subset6SYMG.treeLikelihood  -5933.73410000 10282.5876
## 107         Subset7K80G.treeLikelihood  -3700.27180000 11552.6385
## 108         Subset8K80G.treeLikelihood  -9197.00450000 12457.3059
## 109        Subset9HKYGX.treeLikelihood  -3783.68050000  8926.9213
## 110      Subset10TRNEFG.treeLikelihood  -4401.77360000  7695.6777
## 111         Subset11JCG.treeLikelihood   -997.18140000 17236.3680
## 112      Subset12TRNEFG.treeLikelihood  -2715.88600000 12128.8444
## 113                        branchRates      0.00000000 25000.0000
## 114                         speciation   -575.32740000   320.6261

Since the ESS values exceeded 200 for all parameters except one (tmrca(Eupercaria)), the maximum clade credibility tree with 95% HPD intervals for all node ages was constructed as a summary of the posterior distribution:

/home/linuxbrew/.linuxbrew/bin/treeannotator -burnin 50000000 -heights median fixed-local-strict-13-calib.trees fixed-local-strict-13-calib.tre

This tree was compared to a preliminary tree based on ~253 million generations to see if the estimated node ages (and that of Eupercaria in particular) changed substantially:

Comparison between the preliminary and final fixed local clock trees (13 calibrations)

Comparison between the preliminary and final fixed local clock trees (13 calibrations)

The 29-calibration version of the fixed local clock analysis was run using the following command, with all other settings identical to the 13-calibration analysis described above:

The analysis was run as follows:

/home/linuxbrew/.linuxbrew/bin/beast -beagle_SSE -beagle_scaling dynamic fixed-local-strict-29-calib.xml

The chain stopped when it reached the specificed length after 12.212 days.

The LogAnalyser-generated summary is shown below:

/home/linuxbrew/.linuxbrew/bin/loganalyser -burnin 50000000 -ess fixed-local-strict-29-calib.log
##                              statistic          median        ESS
## 1                            posterior -49254.91010000  2434.0981
## 2                                prior   -802.45410000   343.2358
## 3                           likelihood -48452.15530000  6045.9048
## 4                 treeModel.rootHeight    142.59460000 17076.6986
## 5                    tmrca(Aipichthys)    142.03190000 16781.5027
## 6              tmrca(Archaeotetraodon)     35.35890000   658.4703
## 7                      tmrca(Archaeus)     55.07860000 14127.8210
## 8                 tmrca(Berybolcensis)     71.52740000 13329.2519
## 9                     tmrca(Calatomus)     37.72320000   940.5783
## 10                   tmrca(Carangaria)     73.59020000   489.6684
## 11          tmrca(Carapus-Lepophidium)     59.88640000 14600.4244
## 12               tmrca(Chaetodontidae)     36.07980000  5150.9271
## 13                     tmrca(Cretzeus)    126.86700000 16617.4014
## 14                tmrca(Eastmanalepes)     50.21910000 16540.2405
## 15                     tmrca(Eobothus)     69.64970000   564.8522
## 16                  tmrca(Eobuglossus)     54.53650000  1514.1785
## 17                  tmrca(Eocoelopoma)     56.48070000 15252.7726
## 18                   tmrca(Eupercaria)     77.52990000   186.9551
## 19            tmrca(Gasterorhamphosus)     72.36050000  1547.8148
## 20                     tmrca(Gobiaria)     60.76200000   477.4315
## 21               tmrca(Homonotichthys)    127.35160000 17241.2467
## 22                  tmrca(Hoplopteryx)    121.19720000  3847.2492
## 23                      tmrca(Luvarus)     67.25050000  2017.7517
## 24               tmrca(Mahengechromis)     51.28850000  1832.4278
## 25              tmrca(Massamorichthys)     58.92270000  4292.1376
## 26                         tmrca(Mene)     60.14150000  5179.8827
## 27                  tmrca(Ovalentaria)     61.76030000  1363.4852
## 28                   tmrca(Pelagiaria)     77.59100000   806.6438
## 29                tmrca(Proacanthurus)     49.37680000  8132.0320
## 30              tmrca(Prosolenostomus)     70.48130000  2028.6770
## 31              tmrca(Rhamphexocoetus)     56.62920000  2183.8936
## 32                tmrca(Rhinocephalus)     60.47450000 15964.6785
## 33  tmrca(Rondeletiidae-Holocentridae)    103.62580000 11596.3732
## 34                         tmrca(Root)    142.59460000 17076.6986
## 35                    tmrca(Sphyraena)     67.29320000  2291.1410
## 36              tmrca(Syngnathiformes)     72.36050000  1547.8148
## 37                       tmrca(Tarkus)     52.32670000   567.8049
## 38                 tmrca(Trachipterus)     22.14160000 17507.0383
## 39                      tmrca(Triodon)     66.31820000   211.1453
## 40                     tmrca(Turkmene)     54.92690000  7314.7810
## 41                     tmrca(Zenopsis)     32.30250000  3187.7425
## 42           tmrca(Paracanthopterygii)    142.03190000 16781.5027
## 43           birthDeath.meanGrowthRate      0.01820000  7719.3617
## 44        birthDeath.relativeDeathRate      0.05020000 17687.3529
## 45                  Subset1HKYGX.kappa      3.21390000 18001.0000
## 46           Subset1HKYGX.frequencies1      0.38520000 17217.6627
## 47           Subset1HKYGX.frequencies2      0.23190000 17965.1316
## 48           Subset1HKYGX.frequencies3      0.15100000 18001.0000
## 49           Subset1HKYGX.frequencies4      0.23040000 18001.0000
## 50                  Subset1HKYGX.alpha      1.05020000 18001.0000
## 51                Subset2TRNEFG.kappa1      4.69450000 17363.5582
## 52                Subset2TRNEFG.kappa2      1.76780000 17993.2732
## 53                 Subset2TRNEFG.alpha      0.68610000 17740.4138
## 54                Subset3TRNEFG.kappa1      3.02860000 18001.0000
## 55                Subset3TRNEFG.kappa2      4.82050000 15075.2321
## 56                 Subset3TRNEFG.alpha      0.47340000 17723.5243
## 57                    Subset4GTRIGX.ac      0.18880000 18001.0000
## 58                    Subset4GTRIGX.ag      0.42650000 18001.0000
## 59                    Subset4GTRIGX.at      0.10000000 17599.1877
## 60                    Subset4GTRIGX.cg      0.37270000 17656.1057
## 61                    Subset4GTRIGX.gt      0.14980000 18001.0000
## 62          Subset4GTRIGX.frequencies1      0.30550000 17970.4523
## 63          Subset4GTRIGX.frequencies2      0.20480000 18001.0000
## 64          Subset4GTRIGX.frequencies3      0.31060000 16904.8408
## 65          Subset4GTRIGX.frequencies4      0.17800000 17955.3471
## 66                 Subset4GTRIGX.alpha      0.45400000 17373.4553
## 67                  Subset5K80IG.kappa      3.52170000 17830.6014
## 68                  Subset5K80IG.alpha      0.32450000 18001.0000
## 69                      Subset6SYMG.ac      0.33480000 17984.7861
## 70                      Subset6SYMG.ag      1.58200000 17645.0258
## 71                      Subset6SYMG.at      0.24540000 18001.0000
## 72                      Subset6SYMG.cg      1.24950000 17698.9665
## 73                      Subset6SYMG.gt      0.26100000 18001.0000
## 74                   Subset6SYMG.alpha      0.52600000 17529.0012
## 75                   Subset7K80G.kappa      2.83670000 18001.0000
## 76                   Subset7K80G.alpha      0.74340000 17117.0746
## 77                   Subset8K80G.kappa      3.12880000 17246.0245
## 78                   Subset8K80G.alpha      1.25610000 17487.3237
## 79                  Subset9HKYGX.kappa      3.53310000 17957.8614
## 80           Subset9HKYGX.frequencies1      0.21290000 18001.0000
## 81           Subset9HKYGX.frequencies2      0.25980000 18001.0000
## 82           Subset9HKYGX.frequencies3      0.30090000 18001.0000
## 83           Subset9HKYGX.frequencies4      0.22560000 18001.0000
## 84                  Subset9HKYGX.alpha      0.64580000 16997.3722
## 85               Subset10TRNEFG.kappa1      4.12010000 18001.0000
## 86               Subset10TRNEFG.kappa2      5.82070000 18001.0000
## 87                Subset10TRNEFG.alpha      0.52140000 18001.0000
## 88                   Subset11JCG.alpha      0.38510000 18001.0000
## 89               Subset12TRNEFG.kappa1      3.00570000 17780.5387
## 90               Subset12TRNEFG.kappa2      6.86550000 17692.5054
## 91                Subset12TRNEFG.alpha      0.36060000 17033.8502
## 92                     Subset1HKYGX.mu      1.14460000 18001.0000
## 93                    Subset2TRNEFG.mu      0.87870000 17710.6125
## 94                    Subset3TRNEFG.mu      0.29700000 18001.0000
## 95                    Subset4GTRIGX.mu      0.73490000 17883.7564
## 96                     Subset5K80IG.mu      0.82610000 18001.0000
## 97                      Subset6SYMG.mu      1.69220000 18001.0000
## 98                      Subset7K80G.mu      1.85250000 18001.0000
## 99                      Subset8K80G.mu      1.06560000 17875.7495
## 100                    Subset9HKYGX.mu      0.59680000 16795.9376
## 101                  Subset10TRNEFG.mu      1.18470000 18001.0000
## 102                     Subset11JCG.mu      0.21330000 17663.5206
## 103                  Subset12TRNEFG.mu      2.40260000 18001.0000
## 104                         clock.rate      0.00060673  6822.7059
## 105                    Carangaria.rate      0.00036147  1579.6530
## 106           Carapus-Lepophidium.rate      0.00297940 18001.0000
## 107                    Eupercaria.rate      0.00054166   416.9227
## 108                      Gobiaria.rate      0.00167370   490.1731
## 109                   Ovalentaria.rate      0.00047408  1929.6305
## 110                    Pelagiaria.rate      0.00017121  3161.6316
## 111   Rondeletiidae-Holocentridae.rate      0.00016878 14973.1137
## 112               Syngnathiformes.rate      0.00134710  3713.7027
## 113            Paracanthopterygii.rate      0.00094029 16643.5391
## 114                           meanRate      0.00063772   612.9078
## 115             coefficientOfVariation      0.63710000   590.3531
## 116                         covariance      0.91730000   668.7762
## 117        Subset1HKYGX.treeLikelihood  -1963.34490000 13177.2867
## 118       Subset2TRNEFG.treeLikelihood  -3050.34800000 12504.2159
## 119       Subset3TRNEFG.treeLikelihood  -2316.59780000 14307.2615
## 120       Subset4GTRIGX.treeLikelihood  -4918.48890000 11326.9575
## 121        Subset5K80IG.treeLikelihood  -5439.47450000 11217.2113
## 122         Subset6SYMG.treeLikelihood  -5933.09870000 14748.4056
## 123         Subset7K80G.treeLikelihood  -3702.09060000 13827.7245
## 124         Subset8K80G.treeLikelihood  -9209.33330000 11425.4101
## 125        Subset9HKYGX.treeLikelihood  -3789.74140000 14233.0811
## 126      Subset10TRNEFG.treeLikelihood  -4410.91230000 10183.5781
## 127         Subset11JCG.treeLikelihood   -999.78800000 17085.3841
## 128      Subset12TRNEFG.treeLikelihood  -2717.90640000 13060.8015
## 129                        branchRates      0.00000000 25000.0000
## 130                         speciation   -582.71500000   328.4556

Similar to the 13-calibration analysis, the ESS values of the 29-calibration run also exceeded 200 for all parameters except tmrca(Eupercaria). A summary tree was therefore constructed and compared to a preliminary tree based on the first ~221 million generations:

/home/linuxbrew/.linuxbrew/bin/treeannotator -burnin 50000000 -heights median fixed-local-strict-29-calib.trees fixed-local-strict-29-calib.tre
Comparison between the preliminary and final fixed local clock trees (29 calibrations)

Comparison between the preliminary and final fixed local clock trees (29 calibrations)

Comparison of the BEAST analyses under the fixed local clocks and 12/28 calibrations with the manuscript tree

Comparison of the BEAST analyses under the fixed local clocks and 12/28 calibrations with the manuscript tree

MCMCTree analysis

In PAML, subsets of a concatenated alignment cannot be simply ignored as in BEAST but must be removed from the file, so that the sum of partition lengths equals the total number of nucleotides (PAML manual: p. 13). Moreover, the sites that make up a partition must be adjacent in the alignment. Therefore, the following information from the best_scheme file generated by PartitionFinder was used to generate a PAML-compatible alignment:

1 locus42, locus1
2 locus57, locus3, locus25, locus2
3 locus4
4 locus61, locus5, locus55, locus63, locus23, locus47
5 locus16, locus54, locus45, locus31, locus7, locus64, locus6
6 locus15, locus24, locus8, locus11, locus53, locus52, locus51, locus20, locus26
7 locus14, locus17, locus9, locus27, locus22, locus28
8 locus10, locus21, locus13
9 locus62, locus37, locus59, locus29, locus12, locus58, locus46, locus66, locus44, locus18
10 locus40, locus39, locus49, locus19, locus32, locus38
11 locus35, locus60, locus33, locus43, locus30
12 locus34
13 locus36, locus48, locus65
14 locus56, locus50, locus41
  1. Using bash, replace whitespaces between each taxon name and the corresponding sequence with line breaks:

    xargs -n 1 < concat.phy > concat2.phy
  2. Manually remove the first line indicating the number of taxa and sites.

  3. Now, use the structure of the new file (with names and sequences in alternating rows) to reorganize the chunks:

    concat <- read.table("/Users/David/Downloads/concat2.phy", stringsAsFactors = FALSE)
    
    # Random check, part 1: print "locus 42" (sites 2051--2100) of the first taxon:
    substring(concat[2,], 2051, 2100)
    
    for(i in seq(2, nrow(concat), by = 2)) {
      chunks <- substring(concat[i,], seq(1, 3250, 50), seq(50, 3250, 50))
      chunks[66] <- substring(concat[i,], 3251, 3297)
      ch1 <- paste(chunks[42], chunks[1], sep = "")
      ch2 <- paste(chunks[57], chunks[3], chunks[25], chunks[2], sep = "")
      ch3 <- paste(chunks[61], chunks[5], chunks[55], chunks[63], chunks[23], chunks[47], sep = "")
      ch4 <- paste(chunks[16], chunks[54], chunks[45], chunks[31], chunks[7], chunks[64], chunks[6], sep = "")
      ch5 <- paste(chunks[15], chunks[24], chunks[8], chunks[11], chunks[53], chunks[52], chunks[51], chunks[20], chunks[26], sep = "")
      ch6 <- paste(chunks[14], chunks[17], chunks[9], chunks[27], chunks[22], chunks[28], sep = "")
      ch7 <- paste(chunks[10], chunks[21], chunks[13], sep = "")
      ch8 <- paste(chunks[62], chunks[37], chunks[59], chunks[29], chunks[12], chunks[58], chunks[46], chunks[66], chunks[44], chunks[18], sep = "")
      ch9 <- paste(chunks[40], chunks[39], chunks[49], chunks[19], chunks[32], chunks[38], sep = "")
      ch10 <- paste(chunks[35], chunks[60], chunks[33], chunks[43], chunks[30], sep = "")
      ch11 <- paste(chunks[36], chunks[48], chunks[65], sep = "")
      ch12 <- paste(chunks[56], chunks[50], chunks[41], sep = "")
      concat[i,] <- paste(ch1, ch2, ch3, ch4, ch5, ch6, ch7, ch8, ch9, ch10, ch11, ch12, sep = "")
    }
    
    # Make sure that the new sequence rows have the desired length:
    for(i in seq(2, nrow(concat), by = 2)) {
      print(nchar(concat[i,]))
    }
    
    # Random check, part 2: "locus 42" should now correspond to sites 1--50. Does it?
    substring(concat[2,], 1, 50)
    
    # Yes! Now print the new alignment into a table:
    write.table(concat,
      "/Users/David/Grive/Alfaro_Lab/Acanthomorpha/sate-gblocks-clean-min-90-taxa/Concat/pamlconcat.phy",
      quote = FALSE,
      row.names = FALSE,
      col.names = FALSE)
  4. Add the following two lines to the beginning of the file:

    118 3197 G
    G 12 100 200 300 350 450 300 150 497 300 250 150 150

Unlike BEAST, PAML cannot assign a separate substitution model to each partition, but it is capable of unlinking substitution model parameters across partitions (Warnock et al. 2014: ESM p. 2). Since moderate substitution model overparameterization usually does not pose a problem to Bayesian phylogenetic analyses (Ronquist & Deans 2010), each partition was assigned its own GTR+\(\Gamma\) (“REV”) model. Note that the unlinking of substitution models necessitates the use of empirical (nhomo = 0) rather than estimated base frequencies. Relative rate, equilibrium frequency, and alpha parameters were unlinked across partitions, but branch lengths were kept linked (options Mgene = 4 and Malpha = 1). To facilitate cross-platform comparisons, the root calibration was set to 120.5 Ma (same as the mean of the root prior used in BEAST) in baseml to calculate substitution rates.

The substitution rate estimation finished up in 5:46:48 and yielded the following values:

Partition Rate (subst. per 100 million years)
Gene 1 0.126608
Gene 2 0.087695
Gene 3 0.033200
Gene 4 0.072678
Gene 5 0.072362
Gene 6 0.158487
Gene 7 0.188372
Gene 8 0.107932
Gene 9 0.060550
Gene 10 0.111402
Gene 11 0.026772
Gene 12 0.184536

For MCMCTree, the G option cannot be used, and the partitions must be given as multiple alignments one after another in a single file (http://groups.google.com/forum/#!topic/pamlsoftware/cC7mOgZnNiY). Such a file was compiled as follows:

  1. Run the following code:

    concat <- read.table("/Users/David/Downloads/concat2.phy", stringsAsFactors = FALSE)
    
    alignments <- matrix(nrow = 12*(nrow(concat) + 2), ncol = 1)
    for(i in 1:(nrow(concat)/2)) {
        for(j in 0:11) {
            alignments[(2*i-1) + j*(nrow(concat) + 2), ] <- concat[(2*i-1),]
        }
      chunks <- substring(concat[2*i,], seq(1, 3250, 50), seq(50, 3250, 50))
      chunks[66] <- substring(concat[2*i,], 3251, 3297)
      alignments[2*i,] <- paste(chunks[42], chunks[1], sep = "")
      alignments[2*i + 1*(nrow(concat) + 2),] <- paste(chunks[57], chunks[3], chunks[25], chunks[2], sep = "")
      alignments[2*i + 2*(nrow(concat) + 2),] <- paste(chunks[61], chunks[5], chunks[55], chunks[63], chunks[23], chunks[47], sep = "")
      alignments[2*i + 3*(nrow(concat) + 2),] <- paste(chunks[16], chunks[54], chunks[45], chunks[31], chunks[7], chunks[64], chunks[6], sep = "")
      alignments[2*i + 4*(nrow(concat) + 2),] <- paste(chunks[15], chunks[24], chunks[8], chunks[11], chunks[53], chunks[52], chunks[51], chunks[20], chunks[26], sep = "")
      alignments[2*i + 5*(nrow(concat) + 2),] <- paste(chunks[14], chunks[17], chunks[9], chunks[27], chunks[22], chunks[28], sep = "")
      alignments[2*i + 6*(nrow(concat) + 2),] <- paste(chunks[10], chunks[21], chunks[13], sep = "")
      alignments[2*i + 7*(nrow(concat) + 2),] <- paste(chunks[62], chunks[37], chunks[59], chunks[29], chunks[12], chunks[58], chunks[46], chunks[66], chunks[44], chunks[18], sep = "")
      alignments[2*i + 8*(nrow(concat) + 2),] <- paste(chunks[40], chunks[39], chunks[49], chunks[19], chunks[32], chunks[38], sep = "")
      alignments[2*i + 9*(nrow(concat) + 2),] <- paste(chunks[35], chunks[60], chunks[33], chunks[43], chunks[30], sep = "")
      alignments[2*i + 10*(nrow(concat) + 2),] <- paste(chunks[36], chunks[48], chunks[65], sep = "")
      alignments[2*i + 11*(nrow(concat) + 2),] <- paste(chunks[56], chunks[50], chunks[41], sep = "")
    }
    write.table(alignments, "/Users/David/Downloads/pamlconcatpartitioned.phy", quote = FALSE, row.names = FALSE, col.names = FALSE, na = "")
  2. Manually add the information about the number of taxa and sites in each partition.

To obtain a single estimate that could be used for the gamma-Dirichlet hyperprior on rates (rgene_gamma), a weighted average of these values was computed, with each rate weighted by the length of the corresponding partition:

## [1] 0.09550072

The shape parameter of the gamma-Dirichlet distribution (\(\alpha\)) was set to 2 and the rate parameter (\(\beta\)) was chosen so that the mean (calculated as \(\frac{\alpha}{\beta}\)) was equal to the rate above (expressed as the number of substitutions per 10 million years):

## [1] 209.4225

The hyperprior on the mean of the rate distribution is distributed as follows:

The mean of the gamma hyperprior on the variance of the log rate was set to 0.1 by setting \(\alpha\) equal to 1 and \(\beta\) equal to 10. This corresponds closely to the mean of the ucld.stdev hyperprior in BEAST (0.3 – note that while BEAST places the prior on the standard deviation of the rate distribution, MCMCTree assigns the prior to the variance, or the square of the standard deviation).

The lognormal distribution of rates is plotted below, with the mean and variance (in log-space) set equal to the means of the respective hyperpriors:

Finally, since there are multiple loci, the Dirichlet concentration parameter \(\alpha_D\) was specified and set to the default value of 1, which is described as producing “a reasonable partitioning” in the MCMCTree manual.

The full configuration file is shown below:

          seed = -1
       seqfile = pamlconcatpartitioned.phy
      treefile = 12_cali_no_outgroups_corrected.tre
       outfile = chunks.txt

         ndata = 12                            * Number of partitions
       seqtype = 0                             * Data type: nucleotides
       usedata = 3                             * Store the Hessian matrix for approximate likelihood computation in out.BV
         clock = 2                             * Uncorrelated lognormal relaxed clock
       RootAge = 'B(9.8, 14.3, 1e-300, 0.05)'  * P of less than 98 Ma = 10^(-300) and P of more than 143 Ma = 0.05

         model = 7                             * GTR
         alpha = 0.1                           * Following Alfaro et al.
         ncatG = 8                             * Following Alfaro et al.

     cleandata = 0                             * Do not remove sites with ambiguity

       BDparas = 0.1 0.1 0.01                  * Birth, death, sampling: following Alfaro et al.
   kappa_gamma = 6 2                           * No effect since usedata = 3
   alpha_gamma = 1 1                           * No effect since usedata = 3

   rgene_gamma = 2 209.42 1                    * Gamma-Dirichlet prior on mean rate: estimated using baseml under strict clock
  sigma2_gamma = 1 10 1                        * Gamma-Dirichlet prior on log rate variance

      finetune = 1: .1 .1 .1 .1 .1 .1          * Auto finetune: times, musigma2, rates, mixing, paras, FossilErr

         print = 2                             * Print branch rates into an output file
        burnin = 500000                        * Following Alfaro et al.
      sampfreq = 500                           * Following Alfaro et al.
       nsample = 15000                         * Following Alfaro et al.

The initial analyses (run under the usedata = 3 option to calculate the Hessian matrices for the branch lengths) produced the following warning messages:

xmax = 0.0000e+00 close to zero at 226!
xmax = 0.0000e+00 close to zero at 225!
xmax = 0.0000e+00 close to zero at 223!
xmax = 0.0000e+00 close to zero at 224!

However, these did not cause baseml to crash, and the Hessian matrices were successfully written into out.BV files. Therefore, the usedata variable was set to 2, the out.BV files were moved to in.BV, and four separate MCMC chains with a length of 75 million generations (as specified in the configuration file above) were started.

Test for autocorrelation in PhyloBayes

Rationale: in order to perform Bayes factor model comparisons in PhyloBayes, it is necessary to first run the estbranches program from the Multidistribute package to calculate a variance-covariance matrix for branch lengths. As its input, estbranches requires a substitution model with fixed-value parameters (i.e., the parameters of the model are not estimated simultaneously with the branch lengths). A properly formatted model file can be generated from baseml output using one of the scripts in the package.

  1. Compile paml2modelinf.c from the Multidistribute package v 9/25/03:

    cc paml2modelinf.c arrayutl.c -lm -O -o paml2modelinf
  2. Run baseml on an unpartitioned version of the SortaDate alignment. This was done using a fixed tree topology with a single root calibration at 120.5 Ma (equal to the mean of the uniform distribution with support from 98 to 143 Ma) and the F84 substitution model, which is required by estbranches.

PhyloBayes uses Bayes factors with marginal likelihoods computed using thermodynamic integration (path sampling) for model comparisons. All comparisons are between the user’s model of choice and the default, unconstrained model, whose prior is defined for unrooted trees (Lepage et al. 2007). The commands below were used to perform comparisons between this model and (1) the autocorrelared lognormal model as well as (2) the uncorrelated gamma model, which is the closest equivalent of the uncorrelated models implemented in BEAST (the uncorrelated exponential model offered in BEAUTi is its special case).

Note that the .oes file used for model comparison has the following components:

  1. topology without tip names but with (what appears to be) branch lengths
  2. list of tip names and numbers
  3. list of parent-child triplets (with tips represented by their numbers)
  4. variance-covariance matrix

According to the manual, the variance-covariance matrix can be obtained using Thorne et al.’s estbranches. The PAML manual states that MCMCTree under the usedata = 3 setting performs the same function. Will the PAML-generated matrix be readable by PhyloBayes?

(Since PhyloBayes does not support partitioning, a new PAML analysis was run on an unpartitioned version of the dataset but with otherwise identical settings.)

../bf -cov chunkbrlen.oes -long -ln chunklnbf ../bf -cov chunkbrlen.oes -long -ugam chunkugambf

Uncorrelated gamma vs unconstrained: logBF = 36.6282 [36.4131 : 41.7097] Autocorrelated lognormal vs. unconstrained: logBF = 77.3395 [59.7214 : 78.3891]

Refs:

Lepage T, Bryant D, Philippe H, Lartillot N 2007 A general comparison of relaxed molecular clock models. Mol Biol Evol 24(12): 2669–80

Ronquist F, Deans AR 2010 Bayesian phylogenetics and its influence on insect systematics. Annu Rev Entomol 55: 189–206

Warnock RCM, Parham JF, Joyce WG, Lyson TR, Donoghue PCJ 2014 Calibration uncertainty in molecular dating analyses: there is no substitute for the prior evaluation of time priors. Proc R Soc B 282(1798): 20141013